e2e-kops-grid-cilium-eni-u2004-k26 #1614869963955245056
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 14 failed / 871 succeeded |
| Started | |
| Elapsed | 37m25s |
| Revision | master |
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(filesystem\svolmode\)\]\svolumeMode\sshould\snot\smount\s\/\smap\sunused\svolumes\sin\sa\spod\s\[LinuxOnly\]$'
test/e2e/storage/testsuites/volumemode.go:387 k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeModeTestSuite).DefineTests.func7() test/e2e/storage/testsuites/volumemode.go:387 +0x62bfrom junit_01.xml
[BeforeEach] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/16/23 06:32:56.605�[0m Jan 16 06:32:56.605: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename volumemode �[38;5;243m01/16/23 06:32:56.606�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/16/23 06:32:56.8�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/16/23 06:32:56.928�[0m [BeforeEach] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode test/e2e/framework/metrics/init/init.go:31 [It] should not mount / map unused volumes in a pod [LinuxOnly] test/e2e/storage/testsuites/volumemode.go:354 Jan 16 06:32:57.055: INFO: Creating resource for dynamic PV Jan 16 06:32:57.055: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP:�[0m creating a StorageClass volumemode-1200-e2e-scrhkmf �[38;5;243m01/16/23 06:32:57.056�[0m �[1mSTEP:�[0m creating a claim �[38;5;243m01/16/23 06:32:57.121�[0m �[1mSTEP:�[0m Creating pod �[38;5;243m01/16/23 06:32:57.253�[0m Jan 16 06:32:57.319: INFO: Waiting up to 5m0s for pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe" in namespace "volumemode-1200" to be "running" Jan 16 06:32:57.384: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 64.92082ms Jan 16 06:32:59.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2.12985496s Jan 16 06:33:01.453: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4.133668129s Jan 16 06:33:03.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 6.12946749s Jan 16 06:33:05.454: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 8.134900987s Jan 16 06:33:07.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 10.130005705s Jan 16 06:33:09.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 12.130729164s Jan 16 06:33:11.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 14.129981657s Jan 16 06:33:13.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 16.130209164s Jan 16 06:33:15.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 18.130274262s Jan 16 06:33:17.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 20.130531084s Jan 16 06:33:19.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 22.130281865s Jan 16 06:33:21.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 24.12945375s Jan 16 06:33:23.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 26.130581203s Jan 16 06:33:25.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 28.129811916s Jan 16 06:33:27.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 30.130052614s Jan 16 06:33:29.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 32.130278608s Jan 16 06:33:31.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 34.129591363s Jan 16 06:33:33.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 36.129688884s Jan 16 06:33:35.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 38.130271104s Jan 16 06:33:37.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 40.129633549s Jan 16 06:33:39.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 42.129854405s Jan 16 06:33:41.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 44.129720312s Jan 16 06:33:43.452: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 46.132419475s Jan 16 06:33:45.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 48.130072136s Jan 16 06:33:47.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 50.129966598s Jan 16 06:33:49.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 52.129832552s Jan 16 06:33:51.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 54.130352127s Jan 16 06:33:53.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 56.130057368s Jan 16 06:33:55.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 58.130332279s Jan 16 06:33:57.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.129919931s Jan 16 06:33:59.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.130960933s Jan 16 06:34:01.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.130831965s Jan 16 06:34:03.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.13048645s Jan 16 06:34:05.455: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.135655384s Jan 16 06:34:07.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.130204237s Jan 16 06:34:09.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.13037801s Jan 16 06:34:11.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.12980928s Jan 16 06:34:13.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.129762725s Jan 16 06:34:15.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.130371151s Jan 16 06:34:17.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.129834314s Jan 16 06:34:19.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.129830881s Jan 16 06:34:21.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.130478621s Jan 16 06:34:23.449: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.129913837s Jan 16 06:34:25.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.13006762s Jan 16 06:34:27.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.130575466s Jan 16 06:34:29.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.130935221s Jan 16 06:34:31.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.130720087s Jan 16 06:34:33.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.131036085s Jan 16 06:34:35.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.131611954s Jan 16 06:34:37.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.130752036s Jan 16 06:34:39.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.13098932s Jan 16 06:34:41.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.131235535s Jan 16 06:34:43.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.131162145s Jan 16 06:34:45.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.131159457s Jan 16 06:34:47.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.131208937s Jan 16 06:34:49.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.131035865s Jan 16 06:34:51.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.131364512s Jan 16 06:34:53.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.130801167s Jan 16 06:34:55.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.130871073s Jan 16 06:34:57.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.130820126s Jan 16 06:34:59.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.131120743s Jan 16 06:35:01.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.131648558s Jan 16 06:35:03.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.130980282s Jan 16 06:35:05.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.13119371s Jan 16 06:35:07.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.131752468s Jan 16 06:35:09.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.131541339s Jan 16 06:35:11.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.131097577s Jan 16 06:35:13.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.131192223s Jan 16 06:35:15.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.13144963s Jan 16 06:35:17.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.131076389s Jan 16 06:35:19.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.130915652s Jan 16 06:35:21.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.130960392s Jan 16 06:35:23.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.131386292s Jan 16 06:35:25.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.131226013s Jan 16 06:35:27.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.131133071s Jan 16 06:35:29.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.131687131s Jan 16 06:35:31.456: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.136694171s Jan 16 06:35:33.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.131088925s Jan 16 06:35:35.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.130845389s Jan 16 06:35:37.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.130893126s Jan 16 06:35:39.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.131363733s Jan 16 06:35:41.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.131107367s Jan 16 06:35:43.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.130720084s Jan 16 06:35:45.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.131045075s Jan 16 06:35:47.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.13108834s Jan 16 06:35:49.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.131359976s Jan 16 06:35:51.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.130494s Jan 16 06:35:53.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.130985912s Jan 16 06:35:55.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.130837015s Jan 16 06:35:57.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.131146058s Jan 16 06:35:59.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.130523361s Jan 16 06:36:01.452: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.132165431s Jan 16 06:36:03.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.131061069s Jan 16 06:36:05.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.130916257s Jan 16 06:36:07.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.130688667s Jan 16 06:36:09.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.13063725s Jan 16 06:36:11.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.131613693s Jan 16 06:36:13.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.130987975s Jan 16 06:36:15.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.130646373s Jan 16 06:36:17.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.130400446s Jan 16 06:36:19.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.131293365s Jan 16 06:36:21.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.131562773s Jan 16 06:36:23.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.130915033s Jan 16 06:36:25.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.131371955s Jan 16 06:36:27.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.131904662s Jan 16 06:36:29.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.131108185s Jan 16 06:36:31.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.130666519s Jan 16 06:36:33.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.131017565s Jan 16 06:36:35.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.13063327s Jan 16 06:36:37.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.131356733s Jan 16 06:36:39.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.131353201s Jan 16 06:36:41.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.130548996s Jan 16 06:36:43.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.131671048s Jan 16 06:36:45.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.130763438s Jan 16 06:36:47.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.130824027s Jan 16 06:36:49.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.131471013s Jan 16 06:36:51.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.130594166s Jan 16 06:36:53.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.131465448s Jan 16 06:36:55.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.130715914s Jan 16 06:36:57.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.131266151s Jan 16 06:36:59.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.131521195s Jan 16 06:37:01.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.130720396s Jan 16 06:37:03.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.13107994s Jan 16 06:37:05.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.131191694s Jan 16 06:37:07.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.130920055s Jan 16 06:37:09.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.131054293s Jan 16 06:37:11.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.131259822s Jan 16 06:37:13.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.130898084s Jan 16 06:37:15.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.131703498s Jan 16 06:37:17.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.131157847s Jan 16 06:37:19.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.131050144s Jan 16 06:37:21.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.131506847s Jan 16 06:37:23.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.131204024s Jan 16 06:37:25.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.130901074s Jan 16 06:37:27.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.130585114s Jan 16 06:37:29.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.131158199s Jan 16 06:37:31.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.131243919s Jan 16 06:37:33.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.131092228s Jan 16 06:37:35.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.130570828s Jan 16 06:37:37.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.130944872s Jan 16 06:37:39.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.130864705s Jan 16 06:37:41.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.130982581s Jan 16 06:37:43.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.130778001s Jan 16 06:37:45.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.131104152s Jan 16 06:37:47.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.131241929s Jan 16 06:37:49.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.131238554s Jan 16 06:37:51.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.131058717s Jan 16 06:37:53.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.131187064s Jan 16 06:37:55.451: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.131721186s Jan 16 06:37:57.450: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.130484316s Jan 16 06:37:57.515: INFO: Pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.195756297s Jan 16 06:37:57.516: INFO: Unexpected error: <*pod.timeoutError | 0xc002e252f0>: { msg: "timed out while waiting for pod volumemode-1200/pod-76500f54-a91f-444b-bef4-99021f6b7dfe to be running", observedObjects: [ <*v1.Pod | 0xc0031d3680>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "pod-76500f54-a91f-444b-bef4-99021f6b7dfe", GenerateName: "", Namespace: "volumemode-1200", SelfLink: "", UID: "b6f27e03-693e-4544-9a1d-dc82d653d597", ResourceVersion: "13202", Generation: 0, CreationTimestamp: { Time: { wall: 0, ext: 63809447577, loc: { name: "Local", zone: [ {name: "UTC", offset: 0, isDST: false}, ], tx: [ { when: -576460752303423488, index: 0, isstd: false, isutc: false, }, ], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: "UTC", offset: 0, isDST: false}, }, }, }, DeletionTimestamp: nil, DeletionGracePeriodSeconds: nil, Labels: nil, Annotations: nil, OwnerReferences: nil, Finalizers: nil, ManagedFields: [ { Manager: "e2e.test", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63809447577, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"write-pod\\\"}\":{\".\":{},\"f:command\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:securityContext\":{\".\":{},\"f:privileged\":{}},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{\".\":{},\"f:fsGroup\":{},\"f:seLinuxOptions\":{\".\":{},\"f:level\":{}}},\"f:terminationGracePeriodSeconds\":{},\"f:volumes\":{\".\":{},\"k:{\\\"name\\\":\\\"volume1\\\"}\":{\".\":{},\"f:name\":{},\"f:persistentVolumeClaim\":{\".\":{},\"f:claimName\":{}}}}}}", }, Subresource: "", }, { Manager: "kubelet", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63809447581, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}", }, Subresource: "status", }, ], }, Spec: { Volumes: [ { Name: "volume1", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: { ClaimName: "ebs.csi.aws.comq2wrs", ReadOnly: false, }, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: nil, PortworxVolume: nil, ScaleIO: nil, StorageOS: nil, CSI: nil, Ephemeral: nil, }, }, { Name: "kube-api-access-z9pjn", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Jan 16 06:37:57.516: FAIL: timed out while waiting for pod volumemode-1200/pod-76500f54-a91f-444b-bef4-99021f6b7dfe to be running Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeModeTestSuite).DefineTests.func7() test/e2e/storage/testsuites/volumemode.go:387 +0x62b Jan 16 06:37:57.516: INFO: Deleting pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe" in namespace "volumemode-1200" Jan 16 06:37:57.584: INFO: Wait up to 5m0s for pod "pod-76500f54-a91f-444b-bef4-99021f6b7dfe" to be fully deleted �[1mSTEP:�[0m Deleting pvc �[38;5;243m01/16/23 06:37:59.718�[0m Jan 16 06:37:59.848: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comq2wrs" Jan 16 06:37:59.916: INFO: Waiting up to 5m0s for PersistentVolume pvc-e8908446-a6b0-4fa3-a80a-1527fce636a1 to get deleted Jan 16 06:37:59.982: INFO: PersistentVolume pvc-e8908446-a6b0-4fa3-a80a-1527fce636a1 found and phase=Released (65.313419ms) Jan 16 06:38:05.047: INFO: PersistentVolume pvc-e8908446-a6b0-4fa3-a80a-1527fce636a1 found and phase=Released (5.131048101s) Jan 16 06:38:10.113: INFO: PersistentVolume pvc-e8908446-a6b0-4fa3-a80a-1527fce636a1 was removed �[1mSTEP:�[0m Deleting sc �[38;5;243m01/16/23 06:38:10.114�[0m [AfterEach] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode test/e2e/framework/node/init/init.go:32 Jan 16 06:38:10.180: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/16/23 06:38:10.252�[0m �[1mSTEP:�[0m Collecting events from namespace "volumemode-1200". �[38;5;243m01/16/23 06:38:10.252�[0m �[1mSTEP:�[0m Found 16 events. �[38;5;243m01/16/23 06:38:10.317�[0m Jan 16 06:38:10.317: INFO: At 2023-01-16 06:32:57 +0000 UTC - event for ebs.csi.aws.comq2wrs: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Jan 16 06:38:10.317: INFO: At 2023-01-16 06:32:57 +0000 UTC - event for ebs.csi.aws.comq2wrs: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Jan 16 06:38:10.317: INFO: At 2023-01-16 06:32:57 +0000 UTC - event for ebs.csi.aws.comq2wrs: {ebs.csi.aws.com_i-0bcdb98d646a60a68_d17057b7-c9b5-4040-b078-a410e48d0aff } Provisioning: External provisioner is provisioning volume for claim "volumemode-1200/ebs.csi.aws.comq2wrs" Jan 16 06:38:10.317: INFO: At 2023-01-16 06:33:00 +0000 UTC - event for ebs.csi.aws.comq2wrs: {ebs.csi.aws.com_i-0bcdb98d646a60a68_d17057b7-c9b5-4040-b078-a410e48d0aff } ProvisioningSucceeded: Successfully provisioned volume pvc-e8908446-a6b0-4fa3-a80a-1527fce636a1 Jan 16 06:38:10.317: INFO: At 2023-01-16 06:33:01 +0000 UTC - event for pod-76500f54-a91f-444b-bef4-99021f6b7dfe: {default-scheduler } Scheduled: Successfully assigned volumemode-1200/pod-76500f54-a91f-444b-bef4-99021f6b7dfe to i-0ac9adcf8852f0594 Jan 16 06:38:10.317: INFO: At 2023-01-16 06:33:01 +0000 UTC - event for pod-76500f54-a91f-444b-bef4-99021f6b7dfe: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "64e2ae6101038527d270804e142f3fdcebda04e748a4e482a404c9fa3d008539": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:10.317: INFO: At 2023-01-16 06:33:03 +0000 UTC - event for pod-76500f54-a91f-444b-bef4-99021f6b7dfe: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-e8908446-a6b0-4fa3-a80a-1527fce636a1" Jan 16 06:38:10.317: INFO: At 2023-01-16 06:33:13 +0000 UTC - event for pod-76500f54-a91f-444b-bef4-99021f6b7dfe: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c3704dd769add775df9e99403fc7fb5660a9a34af78f6fcd7757c8455b6f997c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:10.317: INFO: At 2023-01-16 06:33:28 +0000 UTC - event for pod-76500f54-a91f-444b-bef4-99021f6b7dfe: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "2c4817b43915841debb8e9158de6407439a4cdcc636d4881c33fc4b57c482651": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:10.317: INFO: At 2023-01-16 06:33:42 +0000 UTC - event for pod-76500f54-a91f-444b-bef4-99021f6b7dfe: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "807ec1b98de726d29a7474b62c95462d525139273aed1537e8894462fe3e6d53": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:10.317: INFO: At 2023-01-16 06:33:54 +0000 UTC - event for pod-76500f54-a91f-444b-bef4-99021f6b7dfe: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9a6471323d30c58de23d4b4b66b7d99c6be12aabec9550c6a45939879d8a6df6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:10.317: INFO: At 2023-01-16 06:34:07 +0000 UTC - event for pod-76500f54-a91f-444b-bef4-99021f6b7dfe: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "68b1f3fb31345a7ec865c779dafa5b122250480aebaf8b827a16c49eb012dcc2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:10.317: INFO: At 2023-01-16 06:34:22 +0000 UTC - event for pod-76500f54-a91f-444b-bef4-99021f6b7dfe: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f182cb2edb4bc2a67e8f227eb909ca465ef2a2366304e32947baab0204d125b0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:10.317: INFO: At 2023-01-16 06:34:36 +0000 UTC - event for pod-76500f54-a91f-444b-bef4-99021f6b7dfe: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "308bc07bc815b8cb958ecbcb71543ea165a47c6f71313d08173cdb3455da6c2c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:10.317: INFO: At 2023-01-16 06:34:51 +0000 UTC - event for pod-76500f54-a91f-444b-bef4-99021f6b7dfe: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f04c3bf460a53629d0bcb8051dad7a4b92fef5ae2dcc90429b369e40366ba050": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:10.317: INFO: At 2023-01-16 06:35:03 +0000 UTC - event for pod-76500f54-a91f-444b-bef4-99021f6b7dfe: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "215501d1802388c8bc4fb8cb9563e438e7042fd043fd595d80da32f4a5f8dddc": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:10.382: INFO: POD NODE PHASE GRACE CONDITIONS Jan 16 06:38:10.382: INFO: Jan 16 06:38:10.450: INFO: Logging node info for node i-09cbbd5e7917c52c5 Jan 16 06:38:10.515: INFO: Node Info: &Node{ObjectMeta:{i-09cbbd5e7917c52c5 516d50c9-4cfe-4b03-9036-fb7eaefbf36d 15320 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cbbd5e7917c52c5 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cbbd5e7917c52c5 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.43.77 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-3020":"i-09cbbd5e7917c52c5","ebs.csi.aws.com":"i-09cbbd5e7917c52c5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:32:44 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:37:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cbbd5e7917c52c5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:26:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.77,},NodeAddress{Type:ExternalIP,Address:52.25.36.68,},NodeAddress{Type:InternalDNS,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-36-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2589b28acc26043d6f13727e02d530,SystemUUID:ec2589b2-8acc-2604-3d6f-13727e02d530,BootID:f8305071-d74c-44c0-a267-8cc84ea9ad7b,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-3020^9489448c-9567-11ed-b056-f2aad9c35bb6],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-3020^9489448c-9567-11ed-b056-f2aad9c35bb6,DevicePath:,},},Config:nil,},} Jan 16 06:38:10.516: INFO: Logging kubelet events for node i-09cbbd5e7917c52c5 Jan 16 06:38:10.583: INFO: Logging pods the kubelet thinks is on node i-09cbbd5e7917c52c5 Jan 16 06:38:10.723: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:32:40 +0000 UTC (0+7 container statuses recorded) Jan 16 06:38:10.723: INFO: Container csi-attacher ready: true, restart count 0 Jan 16 06:38:10.723: INFO: Container csi-provisioner ready: true, restart count 0 Jan 16 06:38:10.723: INFO: Container csi-resizer ready: true, restart count 0 Jan 16 06:38:10.723: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 16 06:38:10.723: INFO: Container hostpath ready: true, restart count 0 Jan 16 06:38:10.723: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:38:10.723: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:38:10.723: INFO: rs-9ttnx started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container donothing ready: false, restart count 0 Jan 16 06:38:10.723: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nwpjj started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:10.723: INFO: cilium-xxhgc started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:38:10.723: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:38:10.723: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-vvtkb started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:10.723: INFO: hostpath-symlink-prep-provisioning-9295 started at 2023-01-16 06:34:07 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container init-volume-provisioning-9295 ready: false, restart count 0 Jan 16 06:38:10.723: INFO: inline-volume-tester-5b6t4 started at 2023-01-16 06:32:43 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 16 06:38:10.723: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-blsx2 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:10.723: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-pwdrm started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:10.723: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c9wj5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:10.723: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gk8f6 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:10.723: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jmpsk started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:10.723: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-zrtfg started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:10.723: INFO: ebs-csi-node-smxsx started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded) Jan 16 06:38:10.723: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:38:10.723: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:38:10.723: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:38:10.723: INFO: coredns-autoscaler-7cb5c5b969-lt7sb started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container autoscaler ready: true, restart count 0 Jan 16 06:38:10.723: INFO: coredns-559769c974-bxl2p started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container coredns ready: true, restart count 0 Jan 16 06:38:10.723: INFO: csi-mockplugin-0 started at 2023-01-16 06:32:45 +0000 UTC (0+3 container statuses recorded) Jan 16 06:38:10.723: INFO: Container csi-provisioner ready: false, restart count 0 Jan 16 06:38:10.723: INFO: Container driver-registrar ready: false, restart count 0 Jan 16 06:38:10.723: INFO: Container mock ready: false, restart count 0 Jan 16 06:38:10.723: INFO: csi-mockplugin-attacher-0 started at 2023-01-16 06:32:45 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container csi-attacher ready: false, restart count 0 Jan 16 06:38:10.723: INFO: rs-qcmvk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container donothing ready: false, restart count 0 Jan 16 06:38:10.723: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-m6w4z started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:10.723: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0 Jan 16 06:38:10.723: INFO: netserver-0 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:10.723: INFO: Container webserver ready: false, restart count 0 Jan 16 06:38:10.723: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:36:13 +0000 UTC (0+7 container statuses recorded) Jan 16 06:38:10.723: INFO: Container csi-attacher ready: false, restart count 0 Jan 16 06:38:10.723: INFO: Container csi-provisioner ready: false, restart count 0 Jan 16 06:38:10.723: INFO: Container csi-resizer ready: false, restart count 0 Jan 16 06:38:10.723: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 16 06:38:10.723: INFO: Container hostpath ready: false, restart count 0 Jan 16 06:38:10.723: INFO: Container liveness-probe ready: false, restart count 0 Jan 16 06:38:10.723: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 16 06:38:11.028: INFO: Latency metrics for node i-09cbbd5e7917c52c5 Jan 16 06:38:11.028: INFO: Logging node info for node i-09cdc5f20702b22f2 Jan 16 06:38:11.093: INFO: Node Info: &Node{ObjectMeta:{i-09cdc5f20702b22f2 684a930e-dce6-418d-b0e9-bf389ce1a6d1 15164 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cdc5f20702b22f2 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cdc5f20702b22f2 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.120 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cdc5f20702b22f2"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-16 06:37:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cdc5f20702b22f2,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.120,},NodeAddress{Type:ExternalIP,Address:52.25.16.21,},NodeAddress{Type:InternalDNS,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-16-21.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec20ae9de53dd52df737771e3cb612ee,SystemUUID:ec20ae9d-e53d-d52d-f737-771e3cb612ee,BootID:731f2d80-218c-42a6-8438-95aae6d5c60c,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 16 06:38:11.094: INFO: Logging kubelet events for node i-09cdc5f20702b22f2 Jan 16 06:38:11.162: INFO: Logging pods the kubelet thinks is on node i-09cdc5f20702b22f2 Jan 16 06:38:11.237: INFO: hairpin started at 2023-01-16 06:34:50 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container agnhost-container ready: false, restart count 0 Jan 16 06:38:11.237: INFO: all-pods-removed-hgfff started at 2023-01-16 06:33:12 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container c ready: false, restart count 0 Jan 16 06:38:11.237: INFO: all-succeed-pff5k started at 2023-01-16 06:33:56 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container c ready: false, restart count 0 Jan 16 06:38:11.237: INFO: successful-jobs-history-limit-27897515-25xjd started at 2023-01-16 06:35:00 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container c ready: false, restart count 0 Jan 16 06:38:11.237: INFO: netserver-1 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container webserver ready: false, restart count 0 Jan 16 06:38:11.237: INFO: all-succeed-n4z2d started at 2023-01-16 06:34:38 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container c ready: false, restart count 0 Jan 16 06:38:11.237: INFO: rs-99bkg started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container donothing ready: true, restart count 0 Jan 16 06:38:11.237: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5d2hr started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.237: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jh4ht started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.237: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s5kgk started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.237: INFO: all-pods-removed-6lcv7 started at 2023-01-16 06:33:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container c ready: true, restart count 0 Jan 16 06:38:11.237: INFO: hostexec-i-09cdc5f20702b22f2-6w69n started at 2023-01-16 06:34:18 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container agnhost-container ready: true, restart count 0 Jan 16 06:38:11.237: INFO: cilium-wp5d7 started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:38:11.237: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:38:11.237: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ptdxh started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.237: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.237: INFO: all-succeed-wcs5x started at 2023-01-16 06:34:52 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container c ready: false, restart count 0 Jan 16 06:38:11.237: INFO: pod-771554b8-0ed1-4422-82d5-a0a4dc92ec52 started at 2023-01-16 06:34:29 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container write-pod ready: false, restart count 0 Jan 16 06:38:11.237: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7tft7 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.237: INFO: rs-h9gkk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container donothing ready: false, restart count 0 Jan 16 06:38:11.237: INFO: pod-secrets-5bc87a87-80c1-4cdc-bedc-1ae4930ee9bc started at 2023-01-16 06:35:05 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container secret-volume-test ready: false, restart count 0 Jan 16 06:38:11.237: INFO: service-deletion-9hpkv started at 2023-01-16 06:35:33 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container service-deletion ready: false, restart count 0 Jan 16 06:38:11.237: INFO: rs-qzzlk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container donothing ready: false, restart count 0 Jan 16 06:38:11.237: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.237: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-x7cbm started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.237: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-z5z6l started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.237: INFO: pod-client started at 2023-01-16 06:33:06 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container pod-client ready: false, restart count 0 Jan 16 06:38:11.237: INFO: ebs-csi-node-gk7tk started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded) Jan 16 06:38:11.237: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:38:11.237: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:38:11.237: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:38:11.237: INFO: successful-jobs-history-limit-27897516-zff4c started at 2023-01-16 06:36:00 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container c ready: false, restart count 0 Jan 16 06:38:11.237: INFO: successful-jobs-history-limit-27897514-pt4qd started at 2023-01-16 06:34:00 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container c ready: false, restart count 0 Jan 16 06:38:11.237: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2jtsn started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.237: INFO: successful-jobs-history-limit-27897513-n4m26 started at 2023-01-16 06:33:00 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.237: INFO: Container c ready: false, restart count 0 Jan 16 06:38:11.516: INFO: Latency metrics for node i-09cdc5f20702b22f2 Jan 16 06:38:11.516: INFO: Logging node info for node i-0ac9adcf8852f0594 Jan 16 06:38:11.582: INFO: Node Info: &Node{ObjectMeta:{i-0ac9adcf8852f0594 5c88bf26-14b3-4940-8136-fa11c560695e 15325 0 2023-01-16 06:26:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0ac9adcf8852f0594 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-0ac9adcf8852f0594 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.221 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-4883":"i-0ac9adcf8852f0594","ebs.csi.aws.com":"i-0ac9adcf8852f0594"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-16 06:34:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-16 06:37:57 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0ac9adcf8852f0594,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.221,},NodeAddress{Type:ExternalIP,Address:35.90.38.176,},NodeAddress{Type:InternalDNS,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-90-38-176.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2092b4f74388fb9645e6987d21584e,SystemUUID:ec2092b4-f743-88fb-9645-e6987d21584e,BootID:6319a23e-d063-4cef-b18e-f1906415b7a3,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-051cd447c54259a6e kubernetes.io/csi/ebs.csi.aws.com^vol-0c3cdffdafd49e4d9],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-051cd447c54259a6e,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0c3cdffdafd49e4d9,DevicePath:,},},Config:nil,},} Jan 16 06:38:11.582: INFO: Logging kubelet events for node i-0ac9adcf8852f0594 Jan 16 06:38:11.650: INFO: Logging pods the kubelet thinks is on node i-0ac9adcf8852f0594 Jan 16 06:38:11.724: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5656k started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.724: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-crnvb started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.724: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.724: INFO: ebs-csi-node-8bhxv started at 2023-01-16 06:26:07 +0000 UTC (0+3 container statuses recorded) Jan 16 06:38:11.724: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:38:11.724: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:38:11.724: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:38:11.724: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-lbzbs started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.724: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-hlz4f started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.724: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2k7jz started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.724: INFO: pvc-volume-tester-48722 started at 2023-01-16 06:35:01 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container volume-tester ready: false, restart count 0 Jan 16 06:38:11.724: INFO: pod-8becfd69-702e-46ad-bf8e-2de0749dc180 started at 2023-01-16 06:33:46 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container test-container ready: false, restart count 0 Jan 16 06:38:11.724: INFO: liveness-470354a2-716e-4341-b41b-afc9c1378c49 started at 2023-01-16 06:35:01 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container agnhost-container ready: false, restart count 0 Jan 16 06:38:11.724: INFO: rs-vwwhw started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container donothing ready: false, restart count 0 Jan 16 06:38:11.724: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-b9xdt started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.724: INFO: successful-jobs-history-limit-27897517-7lln2 started at 2023-01-16 06:37:00 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container c ready: false, restart count 0 Jan 16 06:38:11.724: INFO: rs-7fgjp started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container donothing ready: true, restart count 0 Jan 16 06:38:11.724: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0 Jan 16 06:38:11.724: INFO: successful-jobs-history-limit-27897518-4trjv started at 2023-01-16 06:38:00 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container c ready: false, restart count 0 Jan 16 06:38:11.724: INFO: all-succeed-b2zfw started at 2023-01-16 06:33:56 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container c ready: false, restart count 0 Jan 16 06:38:11.724: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:34:54 +0000 UTC (0+7 container statuses recorded) Jan 16 06:38:11.724: INFO: Container csi-attacher ready: false, restart count 0 Jan 16 06:38:11.724: INFO: Container csi-provisioner ready: false, restart count 0 Jan 16 06:38:11.724: INFO: Container csi-resizer ready: false, restart count 0 Jan 16 06:38:11.724: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 16 06:38:11.724: INFO: Container hostpath ready: false, restart count 0 Jan 16 06:38:11.724: INFO: Container liveness-probe ready: false, restart count 0 Jan 16 06:38:11.724: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 16 06:38:11.724: INFO: netserver-2 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container webserver ready: true, restart count 0 Jan 16 06:38:11.724: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-w4v5z started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.724: INFO: cilium-9tdp4 started at 2023-01-16 06:26:07 +0000 UTC (1+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:38:11.724: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:38:11.724: INFO: rs-vxcbj started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container donothing ready: false, restart count 0 Jan 16 06:38:11.724: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7xc54 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:11.724: INFO: csi-mockplugin-0 started at 2023-01-16 06:33:16 +0000 UTC (0+3 container statuses recorded) Jan 16 06:38:11.724: INFO: Container csi-provisioner ready: true, restart count 0 Jan 16 06:38:11.724: INFO: Container driver-registrar ready: true, restart count 0 Jan 16 06:38:11.724: INFO: Container mock ready: true, restart count 0 Jan 16 06:38:11.724: INFO: inline-volume-tester-l8h2w started at 2023-01-16 06:34:11 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:11.724: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 16 06:38:12.081: INFO: Latency metrics for node i-0ac9adcf8852f0594 Jan 16 06:38:12.081: INFO: Logging node info for node i-0b3e7c1b3ffabe4e1 Jan 16 06:38:12.147: INFO: Node Info: &Node{ObjectMeta:{i-0b3e7c1b3ffabe4e1 f1f7f96e-8f6d-4178-9a45-c16cedbd8510 13607 0 2023-01-16 06:25:59 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b3e7c1b3ffabe4e1 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-0b3e7c1b3ffabe4e1 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.59.231 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b3e7c1b3ffabe4e1"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:33:15 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:33:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0b3e7c1b3ffabe4e1,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:33:17 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:33:17 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:33:17 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:33:17 +0000 UTC,LastTransitionTime:2023-01-16 06:26:13 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.231,},NodeAddress{Type:ExternalIP,Address:52.12.10.229,},NodeAddress{Type:InternalDNS,Address:i-0b3e7c1b3ffabe4e1.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b3e7c1b3ffabe4e1.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-12-10-229.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec26e5d5e832a48110ff0427b389577a,SystemUUID:ec26e5d5-e832-a481-10ff-0427b389577a,BootID:21853215-8129-4e64-b14a-211cf0ee9b99,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d49f269edca5f14a],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d49f269edca5f14a,DevicePath:,},},Config:nil,},} Jan 16 06:38:12.147: INFO: Logging kubelet events for node i-0b3e7c1b3ffabe4e1 Jan 16 06:38:12.215: INFO: Logging pods the kubelet thinks is on node i-0b3e7c1b3ffabe4e1 Jan 16 06:38:12.286: INFO: cilium-l9bsn started at 2023-01-16 06:26:00 +0000 UTC (1+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:38:12.286: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:38:12.286: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ccx9k started at 2023-01-16 06:32:14 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:12.286: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-fw4ct started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:12.286: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0 Jan 16 06:38:12.286: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5dxgn started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:12.286: INFO: rs-5bvsz started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container donothing ready: true, restart count 0 Jan 16 06:38:12.286: INFO: rs-x97fb started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container donothing ready: true, restart count 0 Jan 16 06:38:12.286: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7z54m started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:12.286: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:12.286: INFO: netserver-3 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container webserver ready: false, restart count 0 Jan 16 06:38:12.286: INFO: coredns-559769c974-v2nv5 started at 2023-01-16 06:26:26 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container coredns ready: true, restart count 0 Jan 16 06:38:12.286: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-4nrmv started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:12.286: INFO: inline-volume-tester-25k5d started at 2023-01-16 06:33:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 16 06:38:12.286: INFO: pod-b3e1f0e9-7784-4982-97d1-edf15e2c898c started at 2023-01-16 06:33:28 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container test-container ready: false, restart count 0 Jan 16 06:38:12.286: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s2299 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:12.286: INFO: ebs-csi-node-zg8kr started at 2023-01-16 06:26:00 +0000 UTC (0+3 container statuses recorded) Jan 16 06:38:12.286: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:38:12.286: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:38:12.286: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:38:12.286: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gqf9j started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:12.286: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nnzrq started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.286: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:12.579: INFO: Latency metrics for node i-0b3e7c1b3ffabe4e1 Jan 16 06:38:12.579: INFO: Logging node info for node i-0bcdb98d646a60a68 Jan 16 06:38:12.645: INFO: Node Info: &Node{ObjectMeta:{i-0bcdb98d646a60a68 ef8446c9-df03-44e4-bfed-57c522916505 14924 0 2023-01-16 06:24:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0bcdb98d646a60a68 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.49.7 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bcdb98d646a60a68"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-16 06:24:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-16 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:24:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:24:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:24:54 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-01-16 06:36:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0bcdb98d646a60a68,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.49.7,},NodeAddress{Type:ExternalIP,Address:35.164.29.62,},NodeAddress{Type:InternalDNS,Address:i-0bcdb98d646a60a68.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0bcdb98d646a60a68.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-164-29-62.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2e02b7de327b42fbf146d4a7087a79,SystemUUID:ec2e02b7-de32-7b42-fbf1-46d4a7087a79,BootID:8ddb4053-2513-4e18-b49d-f322ee6d1f6e,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43640493,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42802003,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967313,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 16 06:38:12.645: INFO: Logging kubelet events for node i-0bcdb98d646a60a68 Jan 16 06:38:12.713: INFO: Logging pods the kubelet thinks is on node i-0bcdb98d646a60a68 Jan 16 06:38:12.784: INFO: etcd-manager-events-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.784: INFO: Container etcd-manager ready: true, restart count 0 Jan 16 06:38:12.784: INFO: kube-controller-manager-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.784: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 16 06:38:12.784: INFO: kube-scheduler-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.784: INFO: Container kube-scheduler ready: true, restart count 0 Jan 16 06:38:12.784: INFO: ebs-csi-node-9sg8r started at 2023-01-16 06:24:42 +0000 UTC (0+3 container statuses recorded) Jan 16 06:38:12.784: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:38:12.784: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:38:12.784: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:38:12.784: INFO: dns-controller-69987775c6-8t6m5 started at 2023-01-16 06:24:43 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.784: INFO: Container dns-controller ready: true, restart count 0 Jan 16 06:38:12.784: INFO: ebs-csi-controller-5bd98b456f-54tj7 started at 2023-01-16 06:24:43 +0000 UTC (0+5 container statuses recorded) Jan 16 06:38:12.784: INFO: Container csi-attacher ready: true, restart count 0 Jan 16 06:38:12.784: INFO: Container csi-provisioner ready: true, restart count 0 Jan 16 06:38:12.784: INFO: Container csi-resizer ready: true, restart count 0 Jan 16 06:38:12.784: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:38:12.784: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:38:12.784: INFO: etcd-manager-main-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.784: INFO: Container etcd-manager ready: true, restart count 0 Jan 16 06:38:12.784: INFO: kube-apiserver-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+2 container statuses recorded) Jan 16 06:38:12.784: INFO: Container healthcheck ready: true, restart count 0 Jan 16 06:38:12.784: INFO: Container kube-apiserver ready: true, restart count 1 Jan 16 06:38:12.784: INFO: cilium-qwpgj started at 2023-01-16 06:24:42 +0000 UTC (1+1 container statuses recorded) Jan 16 06:38:12.784: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:38:12.784: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:38:12.784: INFO: kops-controller-bjgn2 started at 2023-01-16 06:24:42 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.784: INFO: Container kops-controller ready: true, restart count 0 Jan 16 06:38:12.784: INFO: aws-cloud-controller-manager-fb8z2 started at 2023-01-16 06:24:42 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.784: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 16 06:38:12.784: INFO: cilium-operator-5dd44dc49f-sftdz started at 2023-01-16 06:24:43 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:12.784: INFO: Container cilium-operator ready: true, restart count 0 Jan 16 06:38:13.063: INFO: Latency metrics for node i-0bcdb98d646a60a68 [DeferCleanup (Each)] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "volumemode-1200" for this suite. �[38;5;243m01/16/23 06:38:13.063�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-apps\]\sCronJob\sshould\sdelete\ssuccessful\sfinished\sjobs\swith\slimit\sof\sone\ssuccessful\sjob$'
test/e2e/apps/cronjob.go:502
k8s.io/kubernetes/test/e2e/apps.ensureHistoryLimits({0x801e128, 0xc00359cb60}, {0xc0048006e0, 0xc}, 0x75cbe8b?)
test/e2e/apps/cronjob.go:502 +0x7e5
k8s.io/kubernetes/test/e2e/apps.glob..func2.8()
test/e2e/apps/cronjob.go:287 +0xf3
[BeforeEach] [sig-apps] CronJob
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/16/23 06:32:05.844�[0m
Jan 16 06:32:05.844: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename cronjob �[38;5;243m01/16/23 06:32:05.845�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/16/23 06:32:06.037�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/16/23 06:32:06.161�[0m
[BeforeEach] [sig-apps] CronJob
test/e2e/framework/metrics/init/init.go:31
[It] should delete successful finished jobs with limit of one successful job
test/e2e/apps/cronjob.go:280
�[1mSTEP:�[0m Creating an AllowConcurrent cronjob with custom history limit �[38;5;243m01/16/23 06:32:06.285�[0m
�[1mSTEP:�[0m Ensuring a finished job exists �[38;5;243m01/16/23 06:32:06.35�[0m
�[1mSTEP:�[0m Ensuring a finished job exists by listing jobs explicitly �[38;5;243m01/16/23 06:34:06.427�[0m
�[1mSTEP:�[0m Ensuring this job and its pods does not exist anymore �[38;5;243m01/16/23 06:34:06.49�[0m
Jan 16 06:39:06.894: INFO: Unexpected error: Failed to ensure that job does not exists anymore in namespace cronjob-5627:
<*errors.errorString | 0xc000205d60>: {
s: "timed out waiting for the condition",
}
Jan 16 06:39:06.894: FAIL: Failed to ensure that job does not exists anymore in namespace cronjob-5627: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/apps.ensureHistoryLimits({0x801e128, 0xc00359cb60}, {0xc0048006e0, 0xc}, 0x75cbe8b?)
test/e2e/apps/cronjob.go:502 +0x7e5
k8s.io/kubernetes/test/e2e/apps.glob..func2.8()
test/e2e/apps/cronjob.go:287 +0xf3
[AfterEach] [sig-apps] CronJob
test/e2e/framework/node/init/init.go:32
Jan 16 06:39:06.894: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-apps] CronJob
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-apps] CronJob
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/16/23 06:39:06.992�[0m
�[1mSTEP:�[0m Collecting events from namespace "cronjob-5627". �[38;5;243m01/16/23 06:39:06.992�[0m
�[1mSTEP:�[0m Found 73 events. �[38;5;243m01/16/23 06:39:07.118�[0m
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:33:00 +0000 UTC - event for successful-jobs-history-limit: {cronjob-controller } SuccessfulCreate: Created job successful-jobs-history-limit-27897513
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:33:00 +0000 UTC - event for successful-jobs-history-limit-27897513: {job-controller } SuccessfulCreate: Created pod: successful-jobs-history-limit-27897513-n4m26
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:33:00 +0000 UTC - event for successful-jobs-history-limit-27897513-n4m26: {default-scheduler } Scheduled: Successfully assigned cronjob-5627/successful-jobs-history-limit-27897513-n4m26 to i-09cdc5f20702b22f2
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:33:01 +0000 UTC - event for successful-jobs-history-limit-27897513-n4m26: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0720afb0c532f0794b4167e37abf9e32e10ca7907aeb5cc7f3fc29fb896d0a5e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:33:12 +0000 UTC - event for successful-jobs-history-limit-27897513-n4m26: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7154b0abe547711523b87e8d2b90c7b3e70bac159e3de4908241a012220089f5": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:33:24 +0000 UTC - event for successful-jobs-history-limit-27897513-n4m26: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f48bd8a11c05becbaecbc5f199e22994ab446de9bef1018118d0c3062339a03f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:33:38 +0000 UTC - event for successful-jobs-history-limit-27897513-n4m26: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4116143575ae27db447f1d903d4dd7fe963dfd8e69d9940123c03989deb001a8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:33:53 +0000 UTC - event for successful-jobs-history-limit-27897513-n4m26: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "02046bd306e8a21e9c4c889a1df870a690226ee3855df65b767a4e14d1963575": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:34:00 +0000 UTC - event for successful-jobs-history-limit: {cronjob-controller } SuccessfulCreate: Created job successful-jobs-history-limit-27897514
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:34:00 +0000 UTC - event for successful-jobs-history-limit-27897514: {job-controller } SuccessfulCreate: Created pod: successful-jobs-history-limit-27897514-pt4qd
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:34:00 +0000 UTC - event for successful-jobs-history-limit-27897514-pt4qd: {default-scheduler } Scheduled: Successfully assigned cronjob-5627/successful-jobs-history-limit-27897514-pt4qd to i-09cdc5f20702b22f2
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:34:01 +0000 UTC - event for successful-jobs-history-limit-27897514-pt4qd: {kubelet i-09cdc5f20702b22f2} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:34:01 +0000 UTC - event for successful-jobs-history-limit-27897514-pt4qd: {kubelet i-09cdc5f20702b22f2} Created: Created container c
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:34:01 +0000 UTC - event for successful-jobs-history-limit-27897514-pt4qd: {kubelet i-09cdc5f20702b22f2} Started: Started container c
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:34:04 +0000 UTC - event for successful-jobs-history-limit: {cronjob-controller } SawCompletedJob: Saw completed job: successful-jobs-history-limit-27897514, status: Complete
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:34:04 +0000 UTC - event for successful-jobs-history-limit-27897514: {job-controller } Completed: Job completed
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:34:09 +0000 UTC - event for successful-jobs-history-limit-27897513-n4m26: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8c8ed8586c1c91e9282139770f828d9ccef7a9923d6850b362dfbe47a908024d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:34:23 +0000 UTC - event for successful-jobs-history-limit-27897513-n4m26: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0fabfad34fd7e09c116954d7a449334d9c1fac521c9c81e00be013301d9973ed": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:34:34 +0000 UTC - event for successful-jobs-history-limit-27897513-n4m26: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "84d94eb08d181166d02a9ee8f47767c002142f48f44b429cafbef35f46c432a4": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:34:45 +0000 UTC - event for successful-jobs-history-limit-27897513-n4m26: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "da15cbcdd1c6b04f74b198e9e0c0d3462b0115278d3d7bd8644e3c9e22f5dea3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:34:57 +0000 UTC - event for successful-jobs-history-limit-27897513-n4m26: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "340f1d70906189c6cb275116003d85e254f6c275df93c6894df6e62c84e51296": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:35:00 +0000 UTC - event for successful-jobs-history-limit: {cronjob-controller } SuccessfulCreate: Created job successful-jobs-history-limit-27897515
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:35:00 +0000 UTC - event for successful-jobs-history-limit-27897515: {job-controller } SuccessfulCreate: Created pod: successful-jobs-history-limit-27897515-25xjd
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:35:00 +0000 UTC - event for successful-jobs-history-limit-27897515-25xjd: {default-scheduler } Scheduled: Successfully assigned cronjob-5627/successful-jobs-history-limit-27897515-25xjd to i-09cdc5f20702b22f2
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:35:00 +0000 UTC - event for successful-jobs-history-limit-27897515-25xjd: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b73987bd7a8e9383f2337b1f461fa707c39190876f9de8473562c3533e2b5ba3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:35:11 +0000 UTC - event for successful-jobs-history-limit-27897515-25xjd: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f17da73900a42753bc5a6cbfbbf7e4ed3144d78f9c564b9ec36db7b138831a1f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:35:24 +0000 UTC - event for successful-jobs-history-limit-27897515-25xjd: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ef4d9f9e73c6e2f1bd0fe6365b274888d834d57f425b1594bbfbcb226e213093": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:35:37 +0000 UTC - event for successful-jobs-history-limit-27897515-25xjd: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e4c682ec7cef90614c3ddb058955ac980b5076b9ec6287a41a71e5bcb69850e5": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:35:47 +0000 UTC - event for successful-jobs-history-limit-27897515-25xjd: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "70bde0e770ae020b62498dc5c7996a337a17b36d749b08867429fbc484eb133f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:36:00 +0000 UTC - event for successful-jobs-history-limit: {cronjob-controller } SuccessfulCreate: Created job successful-jobs-history-limit-27897516
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:36:00 +0000 UTC - event for successful-jobs-history-limit-27897516: {job-controller } SuccessfulCreate: Created pod: successful-jobs-history-limit-27897516-zff4c
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:36:00 +0000 UTC - event for successful-jobs-history-limit-27897516-zff4c: {default-scheduler } Scheduled: Successfully assigned cronjob-5627/successful-jobs-history-limit-27897516-zff4c to i-09cdc5f20702b22f2
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:36:00 +0000 UTC - event for successful-jobs-history-limit-27897516-zff4c: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "541b4807d54b1a9791f724632e6df68d4c85bf8626c61567af7487179f8d22f3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:36:01 +0000 UTC - event for successful-jobs-history-limit-27897515-25xjd: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e567b6653523c13fa927273a3142fe4124f92d4eab87e62c409e183e33429add": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:36:12 +0000 UTC - event for successful-jobs-history-limit-27897516-zff4c: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "889fbd63f3c7e681e5a1cd9dbadfab3d06f5ace25cdac751d147f343e6c3e7df": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:36:13 +0000 UTC - event for successful-jobs-history-limit-27897515-25xjd: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "12167b8d2194e6bdac668f940bbf437d3ef1fd2e03f39aba9356f5bb32e9b77e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:36:25 +0000 UTC - event for successful-jobs-history-limit-27897515-25xjd: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "757297e7ca7695fa3ce86c68d540f692f3474ef36e50c2dab926d27456490ece": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:36:27 +0000 UTC - event for successful-jobs-history-limit-27897516-zff4c: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "67cffd8daae9f7c4e37de2f91c0bfd17d31416d711e8065f4315ce240ab42fa7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:36:37 +0000 UTC - event for successful-jobs-history-limit-27897516-zff4c: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a3f32f9627580250727d8974f58472b3fd27b51f8173d6f2c47fb3e8c66049dd": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:36:39 +0000 UTC - event for successful-jobs-history-limit-27897515-25xjd: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "16b33f842a0e4460f831970befd503893fda15deca6bbdf7c414096fd7f6fbf5": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:36:50 +0000 UTC - event for successful-jobs-history-limit-27897516-zff4c: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cc4758db3d216dc52e7af2e22722bb7bf233226be6bd469e087ebcfb892872e8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:36:51 +0000 UTC - event for successful-jobs-history-limit-27897515-25xjd: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "85c4ec002a27cb6a8436d749897395a65accd7b820a5eab6b8817685978d451f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:37:00 +0000 UTC - event for successful-jobs-history-limit: {cronjob-controller } SuccessfulCreate: Created job successful-jobs-history-limit-27897517
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:37:00 +0000 UTC - event for successful-jobs-history-limit-27897517: {job-controller } SuccessfulCreate: Created pod: successful-jobs-history-limit-27897517-7lln2
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:37:00 +0000 UTC - event for successful-jobs-history-limit-27897517-7lln2: {default-scheduler } Scheduled: Successfully assigned cronjob-5627/successful-jobs-history-limit-27897517-7lln2 to i-0ac9adcf8852f0594
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:37:00 +0000 UTC - event for successful-jobs-history-limit-27897517-7lln2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "80ef8e3e0477f21b5310dc4c181dabf044b77ec74ee5d82a38fe51051321ab65": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:37:05 +0000 UTC - event for successful-jobs-history-limit-27897516-zff4c: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "094712f40c2cf4f0c4b0bc1ac99de5e6f198c505541b8ccddc39fde700635459": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:37:14 +0000 UTC - event for successful-jobs-history-limit-27897517-7lln2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f20feebeeacadf66eb8a0857de1c35dbc435e6f9c01f7eaea238845ae9ba663e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:37:20 +0000 UTC - event for successful-jobs-history-limit-27897516-zff4c: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "2aa1e5db947974f4d091d0bbddb389910e556de419363e4c1aac94052eb0db71": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:37:27 +0000 UTC - event for successful-jobs-history-limit-27897517-7lln2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0b6c24883982f0278abcfb6e5633f9b379864cb0acc1bcd8743c90c742e4376c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:37:35 +0000 UTC - event for successful-jobs-history-limit-27897516-zff4c: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "54947a711adb5fea68a4e0d15dd528b7c0fa8304e9ec3350a8ef983b2a07cbee": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:37:42 +0000 UTC - event for successful-jobs-history-limit-27897517-7lln2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ed30bb18db5af48824971f6b4a927ab44d51a128fdc4c9fd35dbe5998a180296": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:37:48 +0000 UTC - event for successful-jobs-history-limit-27897516-zff4c: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "abcfade468381b0f99118f75c755720b3fa7574a0491be2d38857d67f69d8ddb": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:37:54 +0000 UTC - event for successful-jobs-history-limit-27897517-7lln2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a0b061936f68c6c353646b0dce971540086a3631f57800439896d7aaa78be8f0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:38:00 +0000 UTC - event for successful-jobs-history-limit: {cronjob-controller } SuccessfulCreate: Created job successful-jobs-history-limit-27897518
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:38:00 +0000 UTC - event for successful-jobs-history-limit-27897518: {job-controller } SuccessfulCreate: Created pod: successful-jobs-history-limit-27897518-4trjv
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:38:00 +0000 UTC - event for successful-jobs-history-limit-27897518-4trjv: {default-scheduler } Scheduled: Successfully assigned cronjob-5627/successful-jobs-history-limit-27897518-4trjv to i-0ac9adcf8852f0594
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:38:00 +0000 UTC - event for successful-jobs-history-limit-27897518-4trjv: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e985e60326f488ae9ecfc6a6f436e986b69f0a2bec40b85670da448eb09be4bd": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:38:01 +0000 UTC - event for successful-jobs-history-limit-27897516-zff4c: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "092ddff38ce9fa1e6cf4627aad14ee502c220426b5d96992904165cfb11227ac": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:38:09 +0000 UTC - event for successful-jobs-history-limit-27897517-7lln2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "de3dccb9f140ded9f29192198886de450517ad7189caae309bcd21ac9ca828a3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:38:11 +0000 UTC - event for successful-jobs-history-limit-27897518-4trjv: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7e6326469c8fef90a0f848db5d95ddffb7e565d791ebac50a2ea2a4011633e6a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:38:22 +0000 UTC - event for successful-jobs-history-limit-27897517-7lln2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "52eb5be011676fd572d329d15f3f74fc4435d9866ce20c67c51f4c670b1813a8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:38:22 +0000 UTC - event for successful-jobs-history-limit-27897518-4trjv: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "36748990ce35cc8023b0d889f76b714f070c2785393edac510c0b4695cb29c9a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:38:35 +0000 UTC - event for successful-jobs-history-limit-27897518-4trjv: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3ccc109813f606e4754b370a2c78d48a4f3dd64328f54a16053848e95fafbda8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:38:36 +0000 UTC - event for successful-jobs-history-limit-27897517-7lln2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "2acb6d513a9dc1bb63eb6c3df532b36612a09ec20ba7254d49b2f1a176bb225a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:38:49 +0000 UTC - event for successful-jobs-history-limit-27897517-7lln2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "47f94ec7597e31bbe4f7700d84d60382766bdbe572c63c43b8ce955e326ef142": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:38:50 +0000 UTC - event for successful-jobs-history-limit-27897518-4trjv: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "666fadc6153bf1c2f5af19388a2b3845855d7caab30ec1a603fc7150b8f4f21b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:39:00 +0000 UTC - event for successful-jobs-history-limit: {cronjob-controller } SuccessfulCreate: Created job successful-jobs-history-limit-27897519
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:39:00 +0000 UTC - event for successful-jobs-history-limit-27897519: {job-controller } SuccessfulCreate: Created pod: successful-jobs-history-limit-27897519-c8xpm
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:39:00 +0000 UTC - event for successful-jobs-history-limit-27897519-c8xpm: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a09391c5a503c66bfd044320197dd0c08009d13a6f55fd23e045c2503b6b0808": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:39:00 +0000 UTC - event for successful-jobs-history-limit-27897519-c8xpm: {default-scheduler } Scheduled: Successfully assigned cronjob-5627/successful-jobs-history-limit-27897519-c8xpm to i-09cdc5f20702b22f2
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:39:01 +0000 UTC - event for successful-jobs-history-limit-27897518-4trjv: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1eb8272645a78dc0a7af70b0ce4950b56dde5095823f6b19e9b70b171b80cbb7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.119: INFO: At 2023-01-16 06:39:02 +0000 UTC - event for successful-jobs-history-limit-27897517-7lln2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "382bd3edbf4eab4b8ac7e6255c1faa392bdc61d4306ac503341fe38922d66d6e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:39:07.183: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 16 06:39:07.183: INFO: successful-jobs-history-limit-27897513-n4m26 i-09cdc5f20702b22f2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:33:00 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:33:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:33:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:33:00 +0000 UTC }]
Jan 16 06:39:07.183: INFO: successful-jobs-history-limit-27897514-pt4qd i-09cdc5f20702b22f2 Succeeded [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:34:00 +0000 UTC PodCompleted } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:34:00 +0000 UTC PodCompleted } {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:34:00 +0000 UTC PodCompleted } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:34:00 +0000 UTC }]
Jan 16 06:39:07.183: INFO: successful-jobs-history-limit-27897515-25xjd i-09cdc5f20702b22f2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:00 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:00 +0000 UTC }]
Jan 16 06:39:07.183: INFO: successful-jobs-history-limit-27897516-zff4c i-09cdc5f20702b22f2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:36:00 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:36:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:36:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:36:00 +0000 UTC }]
Jan 16 06:39:07.183: INFO: successful-jobs-history-limit-27897517-7lln2 i-0ac9adcf8852f0594 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:37:00 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:37:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:37:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:37:00 +0000 UTC }]
Jan 16 06:39:07.183: INFO: successful-jobs-history-limit-27897518-4trjv i-0ac9adcf8852f0594 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:38:00 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:38:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:38:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:38:00 +0000 UTC }]
Jan 16 06:39:07.183: INFO: successful-jobs-history-limit-27897519-c8xpm i-09cdc5f20702b22f2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:39:00 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:39:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:39:00 +0000 UTC ContainersNotReady containers with unready status: [c]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:39:00 +0000 UTC }]
Jan 16 06:39:07.183: INFO:
Jan 16 06:39:07.248: INFO: Unable to fetch cronjob-5627/successful-jobs-history-limit-27897513-n4m26/c logs: the server rejected our request for an unknown reason (get pods successful-jobs-history-limit-27897513-n4m26)
Jan 16 06:39:07.377: INFO: Unable to fetch cronjob-5627/successful-jobs-history-limit-27897515-25xjd/c logs: the server rejected our request for an unknown reason (get pods successful-jobs-history-limit-27897515-25xjd)
Jan 16 06:39:07.441: INFO: Unable to fetch cronjob-5627/successful-jobs-history-limit-27897516-zff4c/c logs: the server rejected our request for an unknown reason (get pods successful-jobs-history-limit-27897516-zff4c)
Jan 16 06:39:07.505: INFO: Unable to fetch cronjob-5627/successful-jobs-history-limit-27897517-7lln2/c logs: the server rejected our request for an unknown reason (get pods successful-jobs-history-limit-27897517-7lln2)
Jan 16 06:39:07.568: INFO:
Logging node info for node i-09cbbd5e7917c52c5
Jan 16 06:39:07.631: INFO: Node Info: &Node{ObjectMeta:{i-09cbbd5e7917c52c5 516d50c9-4cfe-4b03-9036-fb7eaefbf36d 15320 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cbbd5e7917c52c5 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cbbd5e7917c52c5 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.43.77 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-3020":"i-09cbbd5e7917c52c5","ebs.csi.aws.com":"i-09cbbd5e7917c52c5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:32:44 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:37:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cbbd5e7917c52c5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:26:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.77,},NodeAddress{Type:ExternalIP,Address:52.25.36.68,},NodeAddress{Type:InternalDNS,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-36-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2589b28acc26043d6f13727e02d530,SystemUUID:ec2589b2-8acc-2604-3d6f-13727e02d530,BootID:f8305071-d74c-44c0-a267-8cc84ea9ad7b,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-3020^9489448c-9567-11ed-b056-f2aad9c35bb6],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-3020^9489448c-9567-11ed-b056-f2aad9c35bb6,DevicePath:,},},Config:nil,},}
Jan 16 06:39:07.632: INFO:
Logging kubelet events for node i-09cbbd5e7917c52c5
Jan 16 06:39:07.697: INFO:
Logging pods the kubelet thinks is on node i-09cbbd5e7917c52c5
Jan 16 06:39:07.769: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c9wj5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:07.769: INFO: cilium-xxhgc started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:39:07.769: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:39:07.769: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-vvtkb started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:07.769: INFO: hostpath-symlink-prep-provisioning-9295 started at 2023-01-16 06:34:07 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container init-volume-provisioning-9295 ready: false, restart count 0
Jan 16 06:39:07.769: INFO: inline-volume-tester-5b6t4 started at 2023-01-16 06:32:43 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 16 06:39:07.769: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-blsx2 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:07.769: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-pwdrm started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:07.769: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gk8f6 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:07.769: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jmpsk started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:07.769: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-m6w4z started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:07.769: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-zrtfg started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:07.769: INFO: ebs-csi-node-smxsx started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:39:07.769: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:39:07.769: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:39:07.769: INFO: coredns-autoscaler-7cb5c5b969-lt7sb started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container autoscaler ready: true, restart count 0
Jan 16 06:39:07.769: INFO: coredns-559769c974-bxl2p started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container coredns ready: true, restart count 0
Jan 16 06:39:07.769: INFO: rs-qcmvk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container donothing ready: false, restart count 0
Jan 16 06:39:07.769: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0
Jan 16 06:39:07.769: INFO: netserver-0 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container webserver ready: false, restart count 0
Jan 16 06:39:07.769: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:36:13 +0000 UTC (0+7 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container csi-attacher ready: false, restart count 0
Jan 16 06:39:07.769: INFO: Container csi-provisioner ready: false, restart count 0
Jan 16 06:39:07.769: INFO: Container csi-resizer ready: false, restart count 0
Jan 16 06:39:07.769: INFO: Container csi-snapshotter ready: false, restart count 0
Jan 16 06:39:07.769: INFO: Container hostpath ready: false, restart count 0
Jan 16 06:39:07.769: INFO: Container liveness-probe ready: false, restart count 0
Jan 16 06:39:07.769: INFO: Container node-driver-registrar ready: false, restart count 0
Jan 16 06:39:07.769: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:32:40 +0000 UTC (0+7 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container csi-attacher ready: true, restart count 0
Jan 16 06:39:07.769: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:39:07.769: INFO: Container csi-resizer ready: true, restart count 0
Jan 16 06:39:07.769: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 16 06:39:07.769: INFO: Container hostpath ready: true, restart count 0
Jan 16 06:39:07.769: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:39:07.769: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:39:07.769: INFO: rs-9ttnx started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container donothing ready: false, restart count 0
Jan 16 06:39:07.769: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nwpjj started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:07.769: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.038: INFO:
Latency metrics for node i-09cbbd5e7917c52c5
Jan 16 06:39:08.038: INFO:
Logging node info for node i-09cdc5f20702b22f2
Jan 16 06:39:08.101: INFO: Node Info: &Node{ObjectMeta:{i-09cdc5f20702b22f2 684a930e-dce6-418d-b0e9-bf389ce1a6d1 15164 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cdc5f20702b22f2 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cdc5f20702b22f2 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.120 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cdc5f20702b22f2"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-16 06:37:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cdc5f20702b22f2,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.120,},NodeAddress{Type:ExternalIP,Address:52.25.16.21,},NodeAddress{Type:InternalDNS,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-16-21.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec20ae9de53dd52df737771e3cb612ee,SystemUUID:ec20ae9d-e53d-d52d-f737-771e3cb612ee,BootID:731f2d80-218c-42a6-8438-95aae6d5c60c,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 16 06:39:08.101: INFO:
Logging kubelet events for node i-09cdc5f20702b22f2
Jan 16 06:39:08.166: INFO:
Logging pods the kubelet thinks is on node i-09cdc5f20702b22f2
Jan 16 06:39:08.238: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2jtsn started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.238: INFO: successful-jobs-history-limit-27897513-n4m26 started at 2023-01-16 06:33:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container c ready: false, restart count 0
Jan 16 06:39:08.238: INFO: successful-jobs-history-limit-27897514-pt4qd started at 2023-01-16 06:34:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container c ready: false, restart count 0
Jan 16 06:39:08.238: INFO: all-pods-removed-hgfff started at 2023-01-16 06:33:12 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container c ready: false, restart count 0
Jan 16 06:39:08.238: INFO: all-succeed-pff5k started at 2023-01-16 06:33:56 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container c ready: false, restart count 0
Jan 16 06:39:08.238: INFO: hairpin started at 2023-01-16 06:34:50 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container agnhost-container ready: false, restart count 0
Jan 16 06:39:08.238: INFO: all-succeed-n4z2d started at 2023-01-16 06:34:38 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container c ready: false, restart count 0
Jan 16 06:39:08.238: INFO: rs-99bkg started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container donothing ready: true, restart count 0
Jan 16 06:39:08.238: INFO: successful-jobs-history-limit-27897515-25xjd started at 2023-01-16 06:35:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container c ready: false, restart count 0
Jan 16 06:39:08.238: INFO: netserver-1 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container webserver ready: false, restart count 0
Jan 16 06:39:08.238: INFO: implicit-root-uid started at 2023-01-16 06:38:39 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container implicit-root-uid ready: false, restart count 0
Jan 16 06:39:08.238: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5d2hr started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.238: INFO: all-pods-removed-6lcv7 started at 2023-01-16 06:33:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container c ready: true, restart count 0
Jan 16 06:39:08.238: INFO: hostexec-i-09cdc5f20702b22f2-6w69n started at 2023-01-16 06:34:18 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container agnhost-container ready: true, restart count 0
Jan 16 06:39:08.238: INFO: cilium-wp5d7 started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:39:08.238: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:39:08.238: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ptdxh started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.238: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jh4ht started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.238: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s5kgk started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.238: INFO: pod-771554b8-0ed1-4422-82d5-a0a4dc92ec52 started at 2023-01-16 06:34:29 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container write-pod ready: false, restart count 0
Jan 16 06:39:08.238: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7tft7 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.238: INFO: rs-h9gkk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container donothing ready: false, restart count 0
Jan 16 06:39:08.238: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.238: INFO: all-succeed-wcs5x started at 2023-01-16 06:34:52 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container c ready: false, restart count 0
Jan 16 06:39:08.238: INFO: pod-secrets-5bc87a87-80c1-4cdc-bedc-1ae4930ee9bc started at 2023-01-16 06:35:05 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container secret-volume-test ready: false, restart count 0
Jan 16 06:39:08.238: INFO: service-deletion-9hpkv started at 2023-01-16 06:35:33 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container service-deletion ready: false, restart count 0
Jan 16 06:39:08.238: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-z5z6l started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.238: INFO: successful-jobs-history-limit-27897519-c8xpm started at 2023-01-16 06:39:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container c ready: false, restart count 0
Jan 16 06:39:08.238: INFO: ebs-csi-node-gk7tk started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:39:08.238: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:39:08.238: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:39:08.238: INFO: successful-jobs-history-limit-27897516-zff4c started at 2023-01-16 06:36:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container c ready: false, restart count 0
Jan 16 06:39:08.238: INFO: rs-qzzlk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container donothing ready: false, restart count 0
Jan 16 06:39:08.238: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.238: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-x7cbm started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.238: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.490: INFO:
Latency metrics for node i-09cdc5f20702b22f2
Jan 16 06:39:08.490: INFO:
Logging node info for node i-0ac9adcf8852f0594
Jan 16 06:39:08.553: INFO: Node Info: &Node{ObjectMeta:{i-0ac9adcf8852f0594 5c88bf26-14b3-4940-8136-fa11c560695e 15325 0 2023-01-16 06:26:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0ac9adcf8852f0594 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-0ac9adcf8852f0594 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.221 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-4883":"i-0ac9adcf8852f0594","ebs.csi.aws.com":"i-0ac9adcf8852f0594"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-16 06:34:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-16 06:37:57 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0ac9adcf8852f0594,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.221,},NodeAddress{Type:ExternalIP,Address:35.90.38.176,},NodeAddress{Type:InternalDNS,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-90-38-176.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2092b4f74388fb9645e6987d21584e,SystemUUID:ec2092b4-f743-88fb-9645-e6987d21584e,BootID:6319a23e-d063-4cef-b18e-f1906415b7a3,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-051cd447c54259a6e kubernetes.io/csi/ebs.csi.aws.com^vol-0c3cdffdafd49e4d9],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-051cd447c54259a6e,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0c3cdffdafd49e4d9,DevicePath:,},},Config:nil,},}
Jan 16 06:39:08.553: INFO:
Logging kubelet events for node i-0ac9adcf8852f0594
Jan 16 06:39:08.622: INFO:
Logging pods the kubelet thinks is on node i-0ac9adcf8852f0594
Jan 16 06:39:08.695: INFO: rs-vwwhw started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container donothing ready: false, restart count 0
Jan 16 06:39:08.695: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-b9xdt started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.695: INFO: successful-jobs-history-limit-27897517-7lln2 started at 2023-01-16 06:37:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container c ready: false, restart count 0
Jan 16 06:39:08.695: INFO: rs-7fgjp started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container donothing ready: true, restart count 0
Jan 16 06:39:08.695: INFO: pvc-volume-tester-48722 started at 2023-01-16 06:35:01 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container volume-tester ready: false, restart count 0
Jan 16 06:39:08.695: INFO: pod-secrets-ec267146-752d-4009-8f59-fd4a77e1e8fd started at 2023-01-16 06:38:49 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container secret-volume-test ready: false, restart count 0
Jan 16 06:39:08.695: INFO: successful-jobs-history-limit-27897518-4trjv started at 2023-01-16 06:38:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container c ready: false, restart count 0
Jan 16 06:39:08.695: INFO: csi-mockplugin-attacher-0 started at 2023-01-16 06:38:36 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container csi-attacher ready: false, restart count 0
Jan 16 06:39:08.695: INFO: all-succeed-b2zfw started at 2023-01-16 06:33:56 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container c ready: false, restart count 0
Jan 16 06:39:08.695: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:34:54 +0000 UTC (0+7 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container csi-attacher ready: false, restart count 0
Jan 16 06:39:08.695: INFO: Container csi-provisioner ready: false, restart count 0
Jan 16 06:39:08.695: INFO: Container csi-resizer ready: false, restart count 0
Jan 16 06:39:08.695: INFO: Container csi-snapshotter ready: false, restart count 0
Jan 16 06:39:08.695: INFO: Container hostpath ready: false, restart count 0
Jan 16 06:39:08.695: INFO: Container liveness-probe ready: false, restart count 0
Jan 16 06:39:08.695: INFO: Container node-driver-registrar ready: false, restart count 0
Jan 16 06:39:08.695: INFO: csi-mockplugin-0 started at 2023-01-16 06:38:36 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container csi-provisioner ready: false, restart count 0
Jan 16 06:39:08.695: INFO: Container driver-registrar ready: false, restart count 0
Jan 16 06:39:08.695: INFO: Container mock ready: false, restart count 0
Jan 16 06:39:08.695: INFO: cilium-9tdp4 started at 2023-01-16 06:26:07 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:39:08.695: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:39:08.695: INFO: inline-volume-tester-l8h2w started at 2023-01-16 06:34:11 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 16 06:39:08.695: INFO: pod-logs-websocket-f7fd16d2-6330-41d5-bd34-4939ca38a8bc started at 2023-01-16 06:38:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container main ready: false, restart count 0
Jan 16 06:39:08.695: INFO: ebs-csi-node-8bhxv started at 2023-01-16 06:26:07 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:39:08.695: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:39:08.695: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:39:08.695: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-lbzbs started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.695: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5656k started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.695: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.695: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2k7jz started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.695: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0
Jan 16 06:39:08.695: INFO: liveness-470354a2-716e-4341-b41b-afc9c1378c49 started at 2023-01-16 06:35:01 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container agnhost-container ready: false, restart count 0
Jan 16 06:39:08.695: INFO: downwardapi-volume-f3832215-334b-4118-b46d-440628e07a46 started at 2023-01-16 06:38:52 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container client-container ready: false, restart count 0
Jan 16 06:39:08.695: INFO: netserver-2 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container webserver ready: true, restart count 0
Jan 16 06:39:08.695: INFO: rs-vxcbj started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container donothing ready: false, restart count 0
Jan 16 06:39:08.695: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7xc54 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.695: INFO: csi-mockplugin-0 started at 2023-01-16 06:33:16 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:39:08.695: INFO: Container driver-registrar ready: true, restart count 0
Jan 16 06:39:08.695: INFO: Container mock ready: true, restart count 0
Jan 16 06:39:08.695: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-w4v5z started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.695: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-hlz4f started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:08.695: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-crnvb started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:08.695: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:09.034: INFO:
Latency metrics for node i-0ac9adcf8852f0594
Jan 16 06:39:09.034: INFO:
Logging node info for node i-0b3e7c1b3ffabe4e1
Jan 16 06:39:09.097: INFO: Node Info: &Node{ObjectMeta:{i-0b3e7c1b3ffabe4e1 f1f7f96e-8f6d-4178-9a45-c16cedbd8510 15605 0 2023-01-16 06:25:59 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b3e7c1b3ffabe4e1 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-0b3e7c1b3ffabe4e1 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.59.231 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b3e7c1b3ffabe4e1"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:33:15 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:38:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0b3e7c1b3ffabe4e1,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:38:23 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:38:23 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:38:23 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:38:23 +0000 UTC,LastTransitionTime:2023-01-16 06:26:13 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.231,},NodeAddress{Type:ExternalIP,Address:52.12.10.229,},NodeAddress{Type:InternalDNS,Address:i-0b3e7c1b3ffabe4e1.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b3e7c1b3ffabe4e1.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-12-10-229.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec26e5d5e832a48110ff0427b389577a,SystemUUID:ec26e5d5-e832-a481-10ff-0427b389577a,BootID:21853215-8129-4e64-b14a-211cf0ee9b99,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d49f269edca5f14a],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d49f269edca5f14a,DevicePath:,},},Config:nil,},}
Jan 16 06:39:09.097: INFO:
Logging kubelet events for node i-0b3e7c1b3ffabe4e1
Jan 16 06:39:09.162: INFO:
Logging pods the kubelet thinks is on node i-0b3e7c1b3ffabe4e1
Jan 16 06:39:09.233: INFO: ebs-csi-node-zg8kr started at 2023-01-16 06:26:00 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:39:09.233: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:39:09.233: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:39:09.233: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:38:12 +0000 UTC (0+7 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container csi-attacher ready: false, restart count 0
Jan 16 06:39:09.233: INFO: Container csi-provisioner ready: false, restart count 0
Jan 16 06:39:09.233: INFO: Container csi-resizer ready: false, restart count 0
Jan 16 06:39:09.233: INFO: Container csi-snapshotter ready: false, restart count 0
Jan 16 06:39:09.233: INFO: Container hostpath ready: false, restart count 0
Jan 16 06:39:09.233: INFO: Container liveness-probe ready: false, restart count 0
Jan 16 06:39:09.233: INFO: Container node-driver-registrar ready: false, restart count 0
Jan 16 06:39:09.233: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gqf9j started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:09.233: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nnzrq started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:09.233: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s2299 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:09.233: INFO: cilium-l9bsn started at 2023-01-16 06:26:00 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:39:09.233: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:39:09.233: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ccx9k started at 2023-01-16 06:32:14 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:09.233: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-fw4ct started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:09.233: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0
Jan 16 06:39:09.233: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5dxgn started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:09.233: INFO: csi-mockplugin-0 started at 2023-01-16 06:38:22 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container csi-provisioner ready: false, restart count 0
Jan 16 06:39:09.233: INFO: Container driver-registrar ready: false, restart count 0
Jan 16 06:39:09.233: INFO: Container mock ready: false, restart count 0
Jan 16 06:39:09.233: INFO: rs-5bvsz started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container donothing ready: true, restart count 0
Jan 16 06:39:09.233: INFO: rs-x97fb started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container donothing ready: true, restart count 0
Jan 16 06:39:09.233: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7z54m started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:09.233: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:09.233: INFO: csi-mockplugin-attacher-0 started at 2023-01-16 06:38:22 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container csi-attacher ready: false, restart count 0
Jan 16 06:39:09.233: INFO: coredns-559769c974-v2nv5 started at 2023-01-16 06:26:26 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container coredns ready: true, restart count 0
Jan 16 06:39:09.233: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-4nrmv started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:39:09.233: INFO: inline-volume-tester-25k5d started at 2023-01-16 06:33:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 16 06:39:09.233: INFO: netserver-3 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.233: INFO: Container webserver ready: false, restart count 0
Jan 16 06:39:09.504: INFO:
Latency metrics for node i-0b3e7c1b3ffabe4e1
Jan 16 06:39:09.504: INFO:
Logging node info for node i-0bcdb98d646a60a68
Jan 16 06:39:09.567: INFO: Node Info: &Node{ObjectMeta:{i-0bcdb98d646a60a68 ef8446c9-df03-44e4-bfed-57c522916505 14924 0 2023-01-16 06:24:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0bcdb98d646a60a68 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.49.7 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bcdb98d646a60a68"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-16 06:24:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-16 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:24:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:24:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:24:54 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-01-16 06:36:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0bcdb98d646a60a68,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.49.7,},NodeAddress{Type:ExternalIP,Address:35.164.29.62,},NodeAddress{Type:InternalDNS,Address:i-0bcdb98d646a60a68.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0bcdb98d646a60a68.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-164-29-62.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2e02b7de327b42fbf146d4a7087a79,SystemUUID:ec2e02b7-de32-7b42-fbf1-46d4a7087a79,BootID:8ddb4053-2513-4e18-b49d-f322ee6d1f6e,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43640493,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42802003,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967313,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 16 06:39:09.567: INFO:
Logging kubelet events for node i-0bcdb98d646a60a68
Jan 16 06:39:09.632: INFO:
Logging pods the kubelet thinks is on node i-0bcdb98d646a60a68
Jan 16 06:39:09.700: INFO: ebs-csi-controller-5bd98b456f-54tj7 started at 2023-01-16 06:24:43 +0000 UTC (0+5 container statuses recorded)
Jan 16 06:39:09.700: INFO: Container csi-attacher ready: true, restart count 0
Jan 16 06:39:09.700: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:39:09.700: INFO: Container csi-resizer ready: true, restart count 0
Jan 16 06:39:09.700: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:39:09.700: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:39:09.700: INFO: etcd-manager-events-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.700: INFO: Container etcd-manager ready: true, restart count 0
Jan 16 06:39:09.700: INFO: kube-controller-manager-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.700: INFO: Container kube-controller-manager ready: true, restart count 2
Jan 16 06:39:09.700: INFO: kube-scheduler-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.700: INFO: Container kube-scheduler ready: true, restart count 0
Jan 16 06:39:09.700: INFO: ebs-csi-node-9sg8r started at 2023-01-16 06:24:42 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:39:09.700: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:39:09.700: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:39:09.700: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:39:09.700: INFO: dns-controller-69987775c6-8t6m5 started at 2023-01-16 06:24:43 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.700: INFO: Container dns-controller ready: true, restart count 0
Jan 16 06:39:09.700: INFO: cilium-operator-5dd44dc49f-sftdz started at 2023-01-16 06:24:43 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.700: INFO: Container cilium-operator ready: true, restart count 0
Jan 16 06:39:09.700: INFO: etcd-manager-main-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.700: INFO: Container etcd-manager ready: true, restart count 0
Jan 16 06:39:09.700: INFO: kube-apiserver-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+2 container statuses recorded)
Jan 16 06:39:09.700: INFO: Container healthcheck ready: true, restart count 0
Jan 16 06:39:09.700: INFO: Container kube-apiserver ready: true, restart count 1
Jan 16 06:39:09.700: INFO: cilium-qwpgj started at 2023-01-16 06:24:42 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:39:09.700: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:39:09.700: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:39:09.700: INFO: kops-controller-bjgn2 started at 2023-01-16 06:24:42 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.700: INFO: Container kops-controller ready: true, restart count 0
Jan 16 06:39:09.700: INFO: aws-cloud-controller-manager-fb8z2 started at 2023-01-16 06:24:42 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:39:09.700: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 16 06:39:09.954: INFO:
Latency metrics for node i-0bcdb98d646a60a68
[DeferCleanup (Each)] [sig-apps] CronJob
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "cronjob-5627" for this suite. �[38;5;243m01/16/23 06:39:09.954�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-auth\]\sServiceAccounts\sshould\sset\sownership\sand\spermission\swhen\sRunAsUser\sor\sFsGroup\sis\spresent\s\[LinuxOnly\]\s\[NodeFeature\:FSGroup\]$'
test/e2e/framework/pod/output/output.go:237
k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutputMatcher(0x760879d?, {0x7616b36?, 0xc00347be08?}, 0xc001ea9b00, 0x0, {0xc00347be88, 0x4, 0x4}, 0x4?)
test/e2e/framework/pod/output/output.go:237 +0x176
k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutputRegexp(...)
test/e2e/framework/pod/output/output.go:221
k8s.io/kubernetes/test/e2e/auth.glob..func5.5()
test/e2e/auth/service_accounts.go:428 +0x9b5
[BeforeEach] [sig-auth] ServiceAccounts
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/16/23 06:32:11.374�[0m
Jan 16 06:32:11.374: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename svcaccounts �[38;5;243m01/16/23 06:32:11.376�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/16/23 06:32:11.579�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/16/23 06:32:11.71�[0m
[BeforeEach] [sig-auth] ServiceAccounts
test/e2e/framework/metrics/init/init.go:31
[It] should set ownership and permission when RunAsUser or FsGroup is present [LinuxOnly] [NodeFeature:FSGroup]
test/e2e/auth/service_accounts.go:336
�[1mSTEP:�[0m Creating a pod to test service account token: �[38;5;243m01/16/23 06:32:11.842�[0m
Jan 16 06:32:11.917: INFO: Waiting up to 5m0s for pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f" in namespace "svcaccounts-4490" to be "Succeeded or Failed"
Jan 16 06:32:11.989: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 71.808191ms
Jan 16 06:32:14.059: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2.141355128s
Jan 16 06:32:16.058: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4.140505642s
Jan 16 06:32:18.063: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 6.145983191s
Jan 16 06:32:20.056: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 8.138830824s
Jan 16 06:32:22.057: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 10.139584374s
Jan 16 06:32:24.056: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 12.138852695s
Jan 16 06:32:26.056: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 14.138870789s
Jan 16 06:32:28.056: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Succeeded", Reason="", readiness=false. Elapsed: 16.138754442s
�[1mSTEP:�[0m Saw pod success �[38;5;243m01/16/23 06:32:28.056�[0m
Jan 16 06:32:28.056: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f" satisfied condition "Succeeded or Failed"
Jan 16 06:32:28.122: INFO: Trying to get logs from node i-09cdc5f20702b22f2 pod test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f container agnhost-container: <nil>
�[1mSTEP:�[0m delete the pod �[38;5;243m01/16/23 06:32:28.198�[0m
Jan 16 06:32:28.271: INFO: Waiting for pod test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f to disappear
Jan 16 06:32:28.337: INFO: Pod test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f no longer exists
�[1mSTEP:�[0m Creating a pod to test service account token: �[38;5;243m01/16/23 06:32:28.337�[0m
Jan 16 06:32:28.405: INFO: Waiting up to 5m0s for pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f" in namespace "svcaccounts-4490" to be "Succeeded or Failed"
Jan 16 06:32:28.471: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 65.913794ms
Jan 16 06:32:30.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2.133691436s
Jan 16 06:32:32.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4.133224176s
Jan 16 06:32:34.545: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 6.13990242s
Jan 16 06:32:36.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 8.133139822s
Jan 16 06:32:38.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 10.132929641s
Jan 16 06:32:40.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 12.133162034s
Jan 16 06:32:42.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 14.133641682s
Jan 16 06:32:44.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 16.133609531s
Jan 16 06:32:46.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 18.133664656s
Jan 16 06:32:48.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 20.133231992s
Jan 16 06:32:50.656: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 22.250489501s
Jan 16 06:32:52.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 24.132824396s
Jan 16 06:32:54.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 26.132992358s
Jan 16 06:32:56.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 28.132858074s
Jan 16 06:32:58.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 30.132427826s
Jan 16 06:33:00.540: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 32.1349752s
Jan 16 06:33:02.542: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 34.13624346s
Jan 16 06:33:04.542: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 36.136687889s
Jan 16 06:33:06.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 38.133421028s
Jan 16 06:33:08.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 40.132948046s
Jan 16 06:33:10.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 42.132892394s
Jan 16 06:33:12.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 44.132256879s
Jan 16 06:33:14.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 46.133097279s
Jan 16 06:33:16.542: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 48.136112953s
Jan 16 06:33:18.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 50.132912721s
Jan 16 06:33:20.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 52.132406681s
Jan 16 06:33:22.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 54.133077767s
Jan 16 06:33:24.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 56.133544461s
Jan 16 06:33:26.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 58.133189412s
Jan 16 06:33:28.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.13332821s
Jan 16 06:33:30.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.132722695s
Jan 16 06:33:32.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.132411332s
Jan 16 06:33:34.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.133285568s
Jan 16 06:33:36.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.133389415s
Jan 16 06:33:38.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.133296554s
Jan 16 06:33:40.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.133017623s
Jan 16 06:33:42.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.133239538s
Jan 16 06:33:44.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.133240956s
Jan 16 06:33:46.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.132867744s
Jan 16 06:33:48.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.13324014s
Jan 16 06:33:50.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.133246304s
Jan 16 06:33:52.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.133534469s
Jan 16 06:33:54.540: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.134668576s
Jan 16 06:33:56.541: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.135650899s
Jan 16 06:33:58.574: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.168710593s
Jan 16 06:34:00.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.132708072s
Jan 16 06:34:02.541: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.135482735s
Jan 16 06:34:04.959: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.553643797s
Jan 16 06:34:06.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.132691794s
Jan 16 06:34:08.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.133144427s
Jan 16 06:34:10.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.132861132s
Jan 16 06:34:12.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.133144675s
Jan 16 06:34:14.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.133031384s
Jan 16 06:34:16.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.132788347s
Jan 16 06:34:18.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.133205175s
Jan 16 06:34:20.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.133095106s
Jan 16 06:34:22.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.133080641s
Jan 16 06:34:24.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.132955746s
Jan 16 06:34:26.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.133169027s
Jan 16 06:34:28.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.133567074s
Jan 16 06:34:30.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.132906861s
Jan 16 06:34:32.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.133189394s
Jan 16 06:34:34.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.133043928s
Jan 16 06:34:36.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.133297934s
Jan 16 06:34:38.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.133184247s
Jan 16 06:34:40.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.132610834s
Jan 16 06:34:42.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.133263057s
Jan 16 06:34:44.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.133116954s
Jan 16 06:34:46.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.132111351s
Jan 16 06:34:48.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.132878629s
Jan 16 06:34:50.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.132719248s
Jan 16 06:34:52.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.132177405s
Jan 16 06:34:54.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.133500751s
Jan 16 06:34:56.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.133505667s
Jan 16 06:34:58.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.133142454s
Jan 16 06:35:00.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.132718298s
Jan 16 06:35:02.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.133719615s
Jan 16 06:35:04.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.132860155s
Jan 16 06:35:06.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.133182218s
Jan 16 06:35:08.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.132397271s
Jan 16 06:35:10.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.133088092s
Jan 16 06:35:12.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.133669555s
Jan 16 06:35:14.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.133212183s
Jan 16 06:35:16.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.133524823s
Jan 16 06:35:18.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.133418816s
Jan 16 06:35:20.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.132702793s
Jan 16 06:35:22.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.133558106s
Jan 16 06:35:24.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.133683753s
Jan 16 06:35:26.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.132398335s
Jan 16 06:35:28.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.133539949s
Jan 16 06:35:30.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.133299366s
Jan 16 06:35:32.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.133293703s
Jan 16 06:35:34.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.133429422s
Jan 16 06:35:36.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.13322172s
Jan 16 06:35:38.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.132362391s
Jan 16 06:35:40.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.132998528s
Jan 16 06:35:42.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.133181533s
Jan 16 06:35:44.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.132440121s
Jan 16 06:35:46.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.133081329s
Jan 16 06:35:48.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.133246601s
Jan 16 06:35:50.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.132657791s
Jan 16 06:35:52.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.132130898s
Jan 16 06:35:54.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.132436138s
Jan 16 06:35:56.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.133157601s
Jan 16 06:35:58.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.133045849s
Jan 16 06:36:00.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.132819112s
Jan 16 06:36:02.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.133679272s
Jan 16 06:36:04.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.133156436s
Jan 16 06:36:06.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.133435347s
Jan 16 06:36:08.540: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.134225427s
Jan 16 06:36:10.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.132393067s
Jan 16 06:36:12.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.134015136s
Jan 16 06:36:14.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.132603044s
Jan 16 06:36:16.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.132471375s
Jan 16 06:36:18.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.133235967s
Jan 16 06:36:20.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.132463509s
Jan 16 06:36:22.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.133212612s
Jan 16 06:36:24.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.133297059s
Jan 16 06:36:26.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.1327935s
Jan 16 06:36:28.540: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.134378923s
Jan 16 06:36:30.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.132627878s
Jan 16 06:36:32.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.133651638s
Jan 16 06:36:34.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.132499098s
Jan 16 06:36:36.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.133551373s
Jan 16 06:36:38.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.133292113s
Jan 16 06:36:40.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.132816153s
Jan 16 06:36:42.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.132441096s
Jan 16 06:36:44.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.133293063s
Jan 16 06:36:46.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.133179053s
Jan 16 06:36:48.540: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.134278059s
Jan 16 06:36:50.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.132846193s
Jan 16 06:36:52.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.132269392s
Jan 16 06:36:54.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.132553383s
Jan 16 06:36:56.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.133257597s
Jan 16 06:36:58.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.133271147s
Jan 16 06:37:00.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.132747656s
Jan 16 06:37:02.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.132848163s
Jan 16 06:37:04.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.133078289s
Jan 16 06:37:06.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.13271806s
Jan 16 06:37:08.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.132991924s
Jan 16 06:37:10.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.132754111s
Jan 16 06:37:12.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.133324331s
Jan 16 06:37:14.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.133895814s
Jan 16 06:37:16.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.133080251s
Jan 16 06:37:18.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.132917804s
Jan 16 06:37:20.538: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.132723599s
Jan 16 06:37:22.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.133144692s
Jan 16 06:37:24.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.133365315s
Jan 16 06:37:26.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.133213984s
Jan 16 06:37:28.539: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.133299072s
Jan 16 06:37:28.605: INFO: Pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.199197826s
Jan 16 06:37:28.738: INFO: Failed to get logs from node "i-09cdc5f20702b22f2" pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f" container "agnhost-container": the server rejected our request for an unknown reason (get pods test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f)
�[1mSTEP:�[0m delete the pod �[38;5;243m01/16/23 06:37:28.738�[0m
Jan 16 06:37:28.811: INFO: Waiting for pod test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f to disappear
Jan 16 06:37:28.876: INFO: Pod test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f no longer exists
Jan 16 06:37:28.876: INFO: Unexpected error:
<*errors.errorString | 0xc001214fd0>: {
s: "expected pod \"test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f\" success: timed out while waiting for pod svcaccounts-4490/test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f to be Succeeded or Failed",
}
Jan 16 06:37:28.877: FAIL: expected pod "test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f" success: timed out while waiting for pod svcaccounts-4490/test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f to be Succeeded or Failed
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutputMatcher(0x760879d?, {0x7616b36?, 0xc00347be08?}, 0xc001ea9b00, 0x0, {0xc00347be88, 0x4, 0x4}, 0x4?)
test/e2e/framework/pod/output/output.go:237 +0x176
k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutputRegexp(...)
test/e2e/framework/pod/output/output.go:221
k8s.io/kubernetes/test/e2e/auth.glob..func5.5()
test/e2e/auth/service_accounts.go:428 +0x9b5
[AfterEach] [sig-auth] ServiceAccounts
test/e2e/framework/node/init/init.go:32
Jan 16 06:37:28.877: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-auth] ServiceAccounts
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-auth] ServiceAccounts
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/16/23 06:37:28.944�[0m
�[1mSTEP:�[0m Collecting events from namespace "svcaccounts-4490". �[38;5;243m01/16/23 06:37:28.944�[0m
�[1mSTEP:�[0m Found 15 events. �[38;5;243m01/16/23 06:37:29.011�[0m
Jan 16 06:37:29.012: INFO: At 2023-01-16 06:32:11 +0000 UTC - event for test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f: {default-scheduler } Scheduled: Successfully assigned svcaccounts-4490/test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f to i-09cdc5f20702b22f2
Jan 16 06:37:29.012: INFO: At 2023-01-16 06:32:14 +0000 UTC - event for test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f: {kubelet i-09cdc5f20702b22f2} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Jan 16 06:37:29.012: INFO: At 2023-01-16 06:32:14 +0000 UTC - event for test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f: {kubelet i-09cdc5f20702b22f2} Created: Created container agnhost-container
Jan 16 06:37:29.012: INFO: At 2023-01-16 06:32:14 +0000 UTC - event for test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f: {kubelet i-09cdc5f20702b22f2} Started: Started container agnhost-container
Jan 16 06:37:29.012: INFO: At 2023-01-16 06:32:28 +0000 UTC - event for test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f: {default-scheduler } Scheduled: Successfully assigned svcaccounts-4490/test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f to i-09cdc5f20702b22f2
Jan 16 06:37:29.012: INFO: At 2023-01-16 06:32:29 +0000 UTC - event for test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "18ae98f8f0d0176d33a31373666e605dca5aef9efc65b53549121c208c6054de": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:37:29.012: INFO: At 2023-01-16 06:32:40 +0000 UTC - event for test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ca9aeb36b7c493d81dde2041968fb14921c978b414e02373e26ae4f0039700f8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:37:29.012: INFO: At 2023-01-16 06:32:55 +0000 UTC - event for test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f426fcc4531053e6be2155f808f9c0f7b1d3be760d6f956eb75274ca3aa1472c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:37:29.012: INFO: At 2023-01-16 06:33:07 +0000 UTC - event for test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7d71e3415e6ba32c6f88ae2cc53a8c55f5b2e5cfd1212c70c9652ab23298ac30": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:37:29.012: INFO: At 2023-01-16 06:33:18 +0000 UTC - event for test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "43f2e8623d959a3f14484985824d79e5b1d27441154e8079b7ffdc964d63541d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:37:29.012: INFO: At 2023-01-16 06:33:29 +0000 UTC - event for test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6e264c9764dfe79310e493fe29e173ddbd6726c3c0d798b2fe4c6c9813708d3b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:37:29.012: INFO: At 2023-01-16 06:33:41 +0000 UTC - event for test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "51bbef06f78803631fb43751312082773fb8f1c6ff00e3737a43d5dcac995769": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:37:29.012: INFO: At 2023-01-16 06:33:53 +0000 UTC - event for test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5d03e78b186c2ed5cb08207fcf3efde5b06bfac911d63bb790b97943f30ad8b7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:37:29.012: INFO: At 2023-01-16 06:34:07 +0000 UTC - event for test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8991a7baa1e1ade58e9b1fef385458e08d97dd71e2c99f85ab42e377fcfd308a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:37:29.012: INFO: At 2023-01-16 06:34:22 +0000 UTC - event for test-pod-3f06c436-df88-4e7b-a56c-97a30e89b80f: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3af3c7460896db99e806e4a29dc3d39389a2bf39dc82521c87040f1b67a0bc82": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:37:29.077: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 16 06:37:29.077: INFO:
Jan 16 06:37:29.145: INFO:
Logging node info for node i-09cbbd5e7917c52c5
Jan 16 06:37:29.211: INFO: Node Info: &Node{ObjectMeta:{i-09cbbd5e7917c52c5 516d50c9-4cfe-4b03-9036-fb7eaefbf36d 12769 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cbbd5e7917c52c5 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cbbd5e7917c52c5 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.43.77 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-3020":"i-09cbbd5e7917c52c5","ebs.csi.aws.com":"i-09cbbd5e7917c52c5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:32:44 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:32:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cbbd5e7917c52c5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:32:50 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:32:50 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:32:50 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:32:50 +0000 UTC,LastTransitionTime:2023-01-16 06:26:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.77,},NodeAddress{Type:ExternalIP,Address:52.25.36.68,},NodeAddress{Type:InternalDNS,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-36-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2589b28acc26043d6f13727e02d530,SystemUUID:ec2589b2-8acc-2604-3d6f-13727e02d530,BootID:f8305071-d74c-44c0-a267-8cc84ea9ad7b,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-3020^9489448c-9567-11ed-b056-f2aad9c35bb6],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-3020^9489448c-9567-11ed-b056-f2aad9c35bb6,DevicePath:,},},Config:nil,},}
Jan 16 06:37:29.211: INFO:
Logging kubelet events for node i-09cbbd5e7917c52c5
Jan 16 06:37:29.280: INFO:
Logging pods the kubelet thinks is on node i-09cbbd5e7917c52c5
Jan 16 06:37:29.422: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-pwdrm started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.422: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c9wj5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.422: INFO: csi-mockplugin-0 started at 2023-01-16 06:32:16 +0000 UTC (0+4 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container busybox ready: false, restart count 0
Jan 16 06:37:29.422: INFO: Container csi-provisioner ready: false, restart count 0
Jan 16 06:37:29.422: INFO: Container driver-registrar ready: false, restart count 0
Jan 16 06:37:29.422: INFO: Container mock ready: false, restart count 0
Jan 16 06:37:29.422: INFO: cilium-xxhgc started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:37:29.422: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:37:29.422: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-vvtkb started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.422: INFO: hostpath-symlink-prep-provisioning-9295 started at 2023-01-16 06:34:07 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container init-volume-provisioning-9295 ready: false, restart count 0
Jan 16 06:37:29.422: INFO: inline-volume-tester-5b6t4 started at 2023-01-16 06:32:43 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 16 06:37:29.422: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-blsx2 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.422: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gk8f6 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.422: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jmpsk started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.422: INFO: rs-qcmvk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container donothing ready: false, restart count 0
Jan 16 06:37:29.422: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-m6w4z started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.422: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-zrtfg started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.422: INFO: ebs-csi-node-smxsx started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:37:29.422: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:37:29.422: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:37:29.422: INFO: coredns-autoscaler-7cb5c5b969-lt7sb started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container autoscaler ready: true, restart count 0
Jan 16 06:37:29.422: INFO: coredns-559769c974-bxl2p started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container coredns ready: true, restart count 0
Jan 16 06:37:29.422: INFO: csi-mockplugin-0 started at 2023-01-16 06:32:45 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container csi-provisioner ready: false, restart count 0
Jan 16 06:37:29.422: INFO: Container driver-registrar ready: false, restart count 0
Jan 16 06:37:29.422: INFO: Container mock ready: false, restart count 0
Jan 16 06:37:29.422: INFO: csi-mockplugin-attacher-0 started at 2023-01-16 06:32:45 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container csi-attacher ready: false, restart count 0
Jan 16 06:37:29.422: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0
Jan 16 06:37:29.422: INFO: netserver-0 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container webserver ready: false, restart count 0
Jan 16 06:37:29.422: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:36:13 +0000 UTC (0+7 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container csi-attacher ready: false, restart count 0
Jan 16 06:37:29.422: INFO: Container csi-provisioner ready: false, restart count 0
Jan 16 06:37:29.422: INFO: Container csi-resizer ready: false, restart count 0
Jan 16 06:37:29.422: INFO: Container csi-snapshotter ready: false, restart count 0
Jan 16 06:37:29.422: INFO: Container hostpath ready: false, restart count 0
Jan 16 06:37:29.422: INFO: Container liveness-probe ready: false, restart count 0
Jan 16 06:37:29.422: INFO: Container node-driver-registrar ready: false, restart count 0
Jan 16 06:37:29.422: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:32:40 +0000 UTC (0+7 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container csi-attacher ready: true, restart count 0
Jan 16 06:37:29.422: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:37:29.422: INFO: Container csi-resizer ready: true, restart count 0
Jan 16 06:37:29.422: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 16 06:37:29.422: INFO: Container hostpath ready: true, restart count 0
Jan 16 06:37:29.422: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:37:29.422: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:37:29.422: INFO: rs-9ttnx started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container donothing ready: false, restart count 0
Jan 16 06:37:29.422: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nwpjj started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.422: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.715: INFO:
Latency metrics for node i-09cbbd5e7917c52c5
Jan 16 06:37:29.715: INFO:
Logging node info for node i-09cdc5f20702b22f2
Jan 16 06:37:29.782: INFO: Node Info: &Node{ObjectMeta:{i-09cdc5f20702b22f2 684a930e-dce6-418d-b0e9-bf389ce1a6d1 15164 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cdc5f20702b22f2 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cdc5f20702b22f2 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.120 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cdc5f20702b22f2"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-16 06:37:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cdc5f20702b22f2,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.120,},NodeAddress{Type:ExternalIP,Address:52.25.16.21,},NodeAddress{Type:InternalDNS,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-16-21.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec20ae9de53dd52df737771e3cb612ee,SystemUUID:ec20ae9d-e53d-d52d-f737-771e3cb612ee,BootID:731f2d80-218c-42a6-8438-95aae6d5c60c,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 16 06:37:29.782: INFO:
Logging kubelet events for node i-09cdc5f20702b22f2
Jan 16 06:37:29.855: INFO:
Logging pods the kubelet thinks is on node i-09cdc5f20702b22f2
Jan 16 06:37:29.931: INFO: all-pods-removed-6lcv7 started at 2023-01-16 06:33:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container c ready: true, restart count 0
Jan 16 06:37:29.931: INFO: hostexec-i-09cdc5f20702b22f2-6w69n started at 2023-01-16 06:34:18 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container agnhost-container ready: true, restart count 0
Jan 16 06:37:29.931: INFO: cilium-wp5d7 started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:37:29.931: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:37:29.931: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ptdxh started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.931: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jh4ht started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.931: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s5kgk started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.931: INFO: pod-771554b8-0ed1-4422-82d5-a0a4dc92ec52 started at 2023-01-16 06:34:29 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container write-pod ready: false, restart count 0
Jan 16 06:37:29.931: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7tft7 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.931: INFO: rs-h9gkk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container donothing ready: false, restart count 0
Jan 16 06:37:29.931: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.931: INFO: all-succeed-wcs5x started at 2023-01-16 06:34:52 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container c ready: false, restart count 0
Jan 16 06:37:29.931: INFO: pod-secrets-5bc87a87-80c1-4cdc-bedc-1ae4930ee9bc started at 2023-01-16 06:35:05 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container secret-volume-test ready: false, restart count 0
Jan 16 06:37:29.931: INFO: service-deletion-9hpkv started at 2023-01-16 06:35:33 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container service-deletion ready: false, restart count 0
Jan 16 06:37:29.931: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-z5z6l started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.931: INFO: pod-client started at 2023-01-16 06:33:06 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container pod-client ready: false, restart count 0
Jan 16 06:37:29.931: INFO: ebs-csi-node-gk7tk started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:37:29.931: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:37:29.931: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:37:29.931: INFO: successful-jobs-history-limit-27897516-zff4c started at 2023-01-16 06:36:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container c ready: false, restart count 0
Jan 16 06:37:29.931: INFO: rs-qzzlk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container donothing ready: false, restart count 0
Jan 16 06:37:29.931: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.931: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-x7cbm started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.931: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2jtsn started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:29.931: INFO: successful-jobs-history-limit-27897513-n4m26 started at 2023-01-16 06:33:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container c ready: false, restart count 0
Jan 16 06:37:29.931: INFO: successful-jobs-history-limit-27897514-pt4qd started at 2023-01-16 06:34:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container c ready: false, restart count 0
Jan 16 06:37:29.931: INFO: all-pods-removed-hgfff started at 2023-01-16 06:33:12 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container c ready: false, restart count 0
Jan 16 06:37:29.931: INFO: all-succeed-pff5k started at 2023-01-16 06:33:56 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container c ready: false, restart count 0
Jan 16 06:37:29.931: INFO: hairpin started at 2023-01-16 06:34:50 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container agnhost-container ready: false, restart count 0
Jan 16 06:37:29.931: INFO: all-succeed-n4z2d started at 2023-01-16 06:34:38 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container c ready: false, restart count 0
Jan 16 06:37:29.931: INFO: rs-99bkg started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container donothing ready: true, restart count 0
Jan 16 06:37:29.931: INFO: successful-jobs-history-limit-27897515-25xjd started at 2023-01-16 06:35:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container c ready: false, restart count 0
Jan 16 06:37:29.931: INFO: netserver-1 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container webserver ready: false, restart count 0
Jan 16 06:37:29.931: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5d2hr started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:29.931: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.247: INFO:
Latency metrics for node i-09cdc5f20702b22f2
Jan 16 06:37:30.247: INFO:
Logging node info for node i-0ac9adcf8852f0594
Jan 16 06:37:30.314: INFO: Node Info: &Node{ObjectMeta:{i-0ac9adcf8852f0594 5c88bf26-14b3-4940-8136-fa11c560695e 14330 0 2023-01-16 06:26:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0ac9adcf8852f0594 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-0ac9adcf8852f0594 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.221 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-4883":"i-0ac9adcf8852f0594","ebs.csi.aws.com":"i-0ac9adcf8852f0594"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:34:47 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:34:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0ac9adcf8852f0594,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.221,},NodeAddress{Type:ExternalIP,Address:35.90.38.176,},NodeAddress{Type:InternalDNS,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-90-38-176.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2092b4f74388fb9645e6987d21584e,SystemUUID:ec2092b4-f743-88fb-9645-e6987d21584e,BootID:6319a23e-d063-4cef-b18e-f1906415b7a3,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-051cd447c54259a6e kubernetes.io/csi/ebs.csi.aws.com^vol-0c3cdffdafd49e4d9],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-051cd447c54259a6e,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0c3cdffdafd49e4d9,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-00fb00edc1f390b8e,DevicePath:,},},Config:nil,},}
Jan 16 06:37:30.314: INFO:
Logging kubelet events for node i-0ac9adcf8852f0594
Jan 16 06:37:30.383: INFO:
Logging pods the kubelet thinks is on node i-0ac9adcf8852f0594
Jan 16 06:37:30.459: INFO: pod-76500f54-a91f-444b-bef4-99021f6b7dfe started at 2023-01-16 06:33:01 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container write-pod ready: false, restart count 0
Jan 16 06:37:30.459: INFO: all-succeed-b2zfw started at 2023-01-16 06:33:56 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container c ready: false, restart count 0
Jan 16 06:37:30.459: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:34:54 +0000 UTC (0+7 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container csi-attacher ready: false, restart count 0
Jan 16 06:37:30.459: INFO: Container csi-provisioner ready: false, restart count 0
Jan 16 06:37:30.459: INFO: Container csi-resizer ready: false, restart count 0
Jan 16 06:37:30.459: INFO: Container csi-snapshotter ready: false, restart count 0
Jan 16 06:37:30.459: INFO: Container hostpath ready: false, restart count 0
Jan 16 06:37:30.459: INFO: Container liveness-probe ready: false, restart count 0
Jan 16 06:37:30.459: INFO: Container node-driver-registrar ready: false, restart count 0
Jan 16 06:37:30.459: INFO: netserver-2 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container webserver ready: true, restart count 0
Jan 16 06:37:30.459: INFO: cilium-9tdp4 started at 2023-01-16 06:26:07 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:37:30.459: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:37:30.459: INFO: rs-vxcbj started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container donothing ready: false, restart count 0
Jan 16 06:37:30.459: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7xc54 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.459: INFO: csi-mockplugin-0 started at 2023-01-16 06:33:16 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:37:30.459: INFO: Container driver-registrar ready: true, restart count 0
Jan 16 06:37:30.459: INFO: Container mock ready: true, restart count 0
Jan 16 06:37:30.459: INFO: inline-volume-tester-l8h2w started at 2023-01-16 06:34:11 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 16 06:37:30.459: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-w4v5z started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.459: INFO: ebs-csi-node-8bhxv started at 2023-01-16 06:26:07 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:37:30.459: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:37:30.459: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:37:30.459: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-lbzbs started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.459: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-hlz4f started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.459: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5656k started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.459: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-crnvb started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.459: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.459: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2k7jz started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.459: INFO: rs-vwwhw started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container donothing ready: false, restart count 0
Jan 16 06:37:30.459: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-b9xdt started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.459: INFO: successful-jobs-history-limit-27897517-7lln2 started at 2023-01-16 06:37:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container c ready: false, restart count 0
Jan 16 06:37:30.459: INFO: rs-7fgjp started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container donothing ready: true, restart count 0
Jan 16 06:37:30.459: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0
Jan 16 06:37:30.459: INFO: pvc-volume-tester-48722 started at 2023-01-16 06:35:01 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container volume-tester ready: false, restart count 0
Jan 16 06:37:30.459: INFO: pod-8becfd69-702e-46ad-bf8e-2de0749dc180 started at 2023-01-16 06:33:46 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container test-container ready: false, restart count 0
Jan 16 06:37:30.459: INFO: liveness-470354a2-716e-4341-b41b-afc9c1378c49 started at 2023-01-16 06:35:01 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.459: INFO: Container agnhost-container ready: false, restart count 0
Jan 16 06:37:30.777: INFO:
Latency metrics for node i-0ac9adcf8852f0594
Jan 16 06:37:30.777: INFO:
Logging node info for node i-0b3e7c1b3ffabe4e1
Jan 16 06:37:30.844: INFO: Node Info: &Node{ObjectMeta:{i-0b3e7c1b3ffabe4e1 f1f7f96e-8f6d-4178-9a45-c16cedbd8510 13607 0 2023-01-16 06:25:59 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b3e7c1b3ffabe4e1 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-0b3e7c1b3ffabe4e1 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.59.231 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b3e7c1b3ffabe4e1"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:33:15 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:33:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0b3e7c1b3ffabe4e1,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:33:17 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:33:17 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:33:17 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:33:17 +0000 UTC,LastTransitionTime:2023-01-16 06:26:13 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.231,},NodeAddress{Type:ExternalIP,Address:52.12.10.229,},NodeAddress{Type:InternalDNS,Address:i-0b3e7c1b3ffabe4e1.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b3e7c1b3ffabe4e1.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-12-10-229.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec26e5d5e832a48110ff0427b389577a,SystemUUID:ec26e5d5-e832-a481-10ff-0427b389577a,BootID:21853215-8129-4e64-b14a-211cf0ee9b99,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d49f269edca5f14a],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d49f269edca5f14a,DevicePath:,},},Config:nil,},}
Jan 16 06:37:30.844: INFO:
Logging kubelet events for node i-0b3e7c1b3ffabe4e1
Jan 16 06:37:30.913: INFO:
Logging pods the kubelet thinks is on node i-0b3e7c1b3ffabe4e1
Jan 16 06:37:30.994: INFO: rs-x97fb started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container donothing ready: true, restart count 0
Jan 16 06:37:30.994: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-fw4ct started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.994: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0
Jan 16 06:37:30.994: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5dxgn started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.994: INFO: rs-5bvsz started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container donothing ready: true, restart count 0
Jan 16 06:37:30.994: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7z54m started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.994: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.994: INFO: inline-volume-tester-25k5d started at 2023-01-16 06:33:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 16 06:37:30.994: INFO: pod-b3e1f0e9-7784-4982-97d1-edf15e2c898c started at 2023-01-16 06:33:28 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container test-container ready: false, restart count 0
Jan 16 06:37:30.994: INFO: netserver-3 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container webserver ready: false, restart count 0
Jan 16 06:37:30.994: INFO: coredns-559769c974-v2nv5 started at 2023-01-16 06:26:26 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container coredns ready: true, restart count 0
Jan 16 06:37:30.994: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-4nrmv started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.994: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gqf9j started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.994: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nnzrq started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.994: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s2299 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:30.994: INFO: ebs-csi-node-zg8kr started at 2023-01-16 06:26:00 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:37:30.994: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:37:30.994: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:37:30.994: INFO: cilium-l9bsn started at 2023-01-16 06:26:00 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:37:30.994: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:37:30.994: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ccx9k started at 2023-01-16 06:32:14 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:30.994: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:37:31.278: INFO:
Latency metrics for node i-0b3e7c1b3ffabe4e1
Jan 16 06:37:31.278: INFO:
Logging node info for node i-0bcdb98d646a60a68
Jan 16 06:37:31.345: INFO: Node Info: &Node{ObjectMeta:{i-0bcdb98d646a60a68 ef8446c9-df03-44e4-bfed-57c522916505 14924 0 2023-01-16 06:24:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0bcdb98d646a60a68 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.49.7 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bcdb98d646a60a68"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-16 06:24:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-16 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:24:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:24:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:24:54 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-01-16 06:36:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0bcdb98d646a60a68,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.49.7,},NodeAddress{Type:ExternalIP,Address:35.164.29.62,},NodeAddress{Type:InternalDNS,Address:i-0bcdb98d646a60a68.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0bcdb98d646a60a68.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-164-29-62.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2e02b7de327b42fbf146d4a7087a79,SystemUUID:ec2e02b7-de32-7b42-fbf1-46d4a7087a79,BootID:8ddb4053-2513-4e18-b49d-f322ee6d1f6e,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43640493,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42802003,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967313,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 16 06:37:31.345: INFO:
Logging kubelet events for node i-0bcdb98d646a60a68
Jan 16 06:37:31.414: INFO:
Logging pods the kubelet thinks is on node i-0bcdb98d646a60a68
Jan 16 06:37:31.487: INFO: dns-controller-69987775c6-8t6m5 started at 2023-01-16 06:24:43 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:31.487: INFO: Container dns-controller ready: true, restart count 0
Jan 16 06:37:31.487: INFO: ebs-csi-controller-5bd98b456f-54tj7 started at 2023-01-16 06:24:43 +0000 UTC (0+5 container statuses recorded)
Jan 16 06:37:31.487: INFO: Container csi-attacher ready: true, restart count 0
Jan 16 06:37:31.487: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:37:31.487: INFO: Container csi-resizer ready: true, restart count 0
Jan 16 06:37:31.487: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:37:31.487: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:37:31.487: INFO: etcd-manager-events-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:31.487: INFO: Container etcd-manager ready: true, restart count 0
Jan 16 06:37:31.487: INFO: kube-controller-manager-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:31.487: INFO: Container kube-controller-manager ready: true, restart count 2
Jan 16 06:37:31.487: INFO: kube-scheduler-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:31.487: INFO: Container kube-scheduler ready: true, restart count 0
Jan 16 06:37:31.487: INFO: ebs-csi-node-9sg8r started at 2023-01-16 06:24:42 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:37:31.487: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:37:31.487: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:37:31.487: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:37:31.487: INFO: aws-cloud-controller-manager-fb8z2 started at 2023-01-16 06:24:42 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:31.487: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 16 06:37:31.487: INFO: cilium-operator-5dd44dc49f-sftdz started at 2023-01-16 06:24:43 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:31.487: INFO: Container cilium-operator ready: true, restart count 0
Jan 16 06:37:31.487: INFO: etcd-manager-main-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:31.487: INFO: Container etcd-manager ready: true, restart count 0
Jan 16 06:37:31.487: INFO: kube-apiserver-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+2 container statuses recorded)
Jan 16 06:37:31.487: INFO: Container healthcheck ready: true, restart count 0
Jan 16 06:37:31.487: INFO: Container kube-apiserver ready: true, restart count 1
Jan 16 06:37:31.487: INFO: cilium-qwpgj started at 2023-01-16 06:24:42 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:37:31.487: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:37:31.487: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:37:31.487: INFO: kops-controller-bjgn2 started at 2023-01-16 06:24:42 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:37:31.487: INFO: Container kops-controller ready: true, restart count 0
Jan 16 06:37:31.765: INFO:
Latency metrics for node i-0bcdb98d646a60a68
[DeferCleanup (Each)] [sig-auth] ServiceAccounts
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "svcaccounts-4490" for this suite. �[38;5;243m01/16/23 06:37:31.765�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sclient\sKubectl\sevents\sshould\sshow\sevent\swhen\spod\sis\screated$'
test/e2e/kubectl/kubectl.go:1969 k8s.io/kubernetes/test/e2e/kubectl.glob..func1.30.1() test/e2e/kubectl/kubectl.go:1969 +0x53dfrom junit_01.xml
[BeforeEach] [sig-cli] Kubectl client set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/16/23 06:42:01.004�[0m Jan 16 06:42:01.004: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename kubectl �[38;5;243m01/16/23 06:42:01.005�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/16/23 06:42:01.201�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/16/23 06:42:01.329�[0m [BeforeEach] [sig-cli] Kubectl client test/e2e/framework/metrics/init/init.go:31 [BeforeEach] [sig-cli] Kubectl client test/e2e/kubectl/kubectl.go:274 [It] should show event when pod is created test/e2e/kubectl/kubectl.go:1945 �[1mSTEP:�[0m running the image registry.k8s.io/e2e-test-images/httpd:2.4.38-4 �[38;5;243m01/16/23 06:42:01.458�[0m Jan 16 06:42:01.458: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/c04c798c-9565-11ed-bd5d-72eebb4d772a/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-4697 run e2e-test-httpd-pod --image=registry.k8s.io/e2e-test-images/httpd:2.4.38-4 --pod-running-timeout=2m0s --labels=run=e2e-test-httpd-pod' Jan 16 06:42:01.741: INFO: stderr: "" Jan 16 06:42:01.741: INFO: stdout: "pod/e2e-test-httpd-pod created\n" �[1mSTEP:�[0m verifying the pod e2e-test-httpd-pod is running �[38;5;243m01/16/23 06:42:01.741�[0m �[1mSTEP:�[0m show started event for this pod �[38;5;243m01/16/23 06:42:21.844�[0m Jan 16 06:42:21.844: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/c04c798c-9565-11ed-bd5d-72eebb4d772a/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-4697 events --for=pod/e2e-test-httpd-pod' Jan 16 06:42:22.136: INFO: stderr: "" Jan 16 06:42:22.136: INFO: stdout: "LAST SEEN TYPE REASON OBJECT MESSAGE\n21s Normal Scheduled Pod/e2e-test-httpd-pod Successfully assigned kubectl-4697/e2e-test-httpd-pod to i-0ac9adcf8852f0594\n20s Warning FailedCreatePodSandBox Pod/e2e-test-httpd-pod Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox \"a43ccaa2803686db6c0dca3378057d47fb0a17bf2f0fe637819b1ae2b9c99b42\": plugin type=\"cilium-cni\" name=\"cilium\" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available\n5s Normal Pulled Pod/e2e-test-httpd-pod Container image \"registry.k8s.io/e2e-test-images/httpd:2.4.38-4\" already present on machine\n5s Normal Created Pod/e2e-test-httpd-pod Created container e2e-test-httpd-pod\n5s Normal Started Pod/e2e-test-httpd-pod Started container e2e-test-httpd-pod\n" �[1mSTEP:�[0m expect not showing any WARNING message except timeouts �[38;5;243m01/16/23 06:42:22.136�[0m Jan 16 06:42:22.136: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/c04c798c-9565-11ed-bd5d-72eebb4d772a/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-4697 events --types=WARNING --for=pod/e2e-test-httpd-pod' Jan 16 06:42:22.428: INFO: stderr: "" Jan 16 06:42:22.428: INFO: stdout: "LAST SEEN TYPE REASON OBJECT MESSAGE\n20s Warning FailedCreatePodSandBox Pod/e2e-test-httpd-pod Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox \"a43ccaa2803686db6c0dca3378057d47fb0a17bf2f0fe637819b1ae2b9c99b42\": plugin type=\"cilium-cni\" name=\"cilium\" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available\n" Jan 16 06:42:22.429: FAIL: unexpected WARNING event fired Full Stack Trace k8s.io/kubernetes/test/e2e/kubectl.glob..func1.30.1() test/e2e/kubectl/kubectl.go:1969 +0x53d [AfterEach] [sig-cli] Kubectl client test/e2e/framework/node/init/init.go:32 Jan 16 06:42:22.429: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-cli] Kubectl client test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-cli] Kubectl client dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/16/23 06:42:22.499�[0m �[1mSTEP:�[0m Collecting events from namespace "kubectl-4697". �[38;5;243m01/16/23 06:42:22.499�[0m �[1mSTEP:�[0m Found 5 events. �[38;5;243m01/16/23 06:42:22.564�[0m Jan 16 06:42:22.564: INFO: At 2023-01-16 06:42:01 +0000 UTC - event for e2e-test-httpd-pod: {default-scheduler } Scheduled: Successfully assigned kubectl-4697/e2e-test-httpd-pod to i-0ac9adcf8852f0594 Jan 16 06:42:22.564: INFO: At 2023-01-16 06:42:02 +0000 UTC - event for e2e-test-httpd-pod: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a43ccaa2803686db6c0dca3378057d47fb0a17bf2f0fe637819b1ae2b9c99b42": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:42:22.564: INFO: At 2023-01-16 06:42:17 +0000 UTC - event for e2e-test-httpd-pod: {kubelet i-0ac9adcf8852f0594} Pulled: Container image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4" already present on machine Jan 16 06:42:22.564: INFO: At 2023-01-16 06:42:17 +0000 UTC - event for e2e-test-httpd-pod: {kubelet i-0ac9adcf8852f0594} Created: Created container e2e-test-httpd-pod Jan 16 06:42:22.564: INFO: At 2023-01-16 06:42:17 +0000 UTC - event for e2e-test-httpd-pod: {kubelet i-0ac9adcf8852f0594} Started: Started container e2e-test-httpd-pod Jan 16 06:42:22.629: INFO: POD NODE PHASE GRACE CONDITIONS Jan 16 06:42:22.629: INFO: e2e-test-httpd-pod i-0ac9adcf8852f0594 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:42:01 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:42:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:42:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:42:01 +0000 UTC }] Jan 16 06:42:22.629: INFO: Jan 16 06:42:22.768: INFO: Logging node info for node i-09cbbd5e7917c52c5 Jan 16 06:42:22.834: INFO: Node Info: &Node{ObjectMeta:{i-09cbbd5e7917c52c5 516d50c9-4cfe-4b03-9036-fb7eaefbf36d 22485 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cbbd5e7917c52c5 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cbbd5e7917c52c5 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.43.77 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cbbd5e7917c52c5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:41:43 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:41:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cbbd5e7917c52c5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:41:41 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:41:41 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:41:41 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:41:41 +0000 UTC,LastTransitionTime:2023-01-16 06:26:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.77,},NodeAddress{Type:ExternalIP,Address:52.25.36.68,},NodeAddress{Type:InternalDNS,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-36-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2589b28acc26043d6f13727e02d530,SystemUUID:ec2589b2-8acc-2604-3d6f-13727e02d530,BootID:f8305071-d74c-44c0-a267-8cc84ea9ad7b,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0907dae3e7c90e3b8],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0907dae3e7c90e3b8,DevicePath:,},},Config:nil,},} Jan 16 06:42:22.834: INFO: Logging kubelet events for node i-09cbbd5e7917c52c5 Jan 16 06:42:22.901: INFO: Logging pods the kubelet thinks is on node i-09cbbd5e7917c52c5 Jan 16 06:42:22.971: INFO: cilium-xxhgc started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded) Jan 16 06:42:22.971: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:42:22.971: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:42:22.971: INFO: sample-webhook-deployment-865554f4d9-82k76 started at 2023-01-16 06:42:21 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:22.971: INFO: Container sample-webhook ready: false, restart count 0 Jan 16 06:42:22.971: INFO: foo-5pxk6 started at <nil> (0+0 container statuses recorded) Jan 16 06:42:22.971: INFO: netserver-0 started at 2023-01-16 06:42:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:22.971: INFO: Container webserver ready: false, restart count 0 Jan 16 06:42:22.971: INFO: ebs-csi-node-smxsx started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded) Jan 16 06:42:22.971: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:42:22.971: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:42:22.971: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:42:22.971: INFO: coredns-autoscaler-7cb5c5b969-lt7sb started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:22.971: INFO: Container autoscaler ready: true, restart count 0 Jan 16 06:42:22.971: INFO: inline-volume-tester-q4bhv started at 2023-01-16 06:40:42 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:22.971: INFO: Container csi-volume-tester ready: true, restart count 0 Jan 16 06:42:22.971: INFO: foo-g4s4z started at <nil> (0+0 container statuses recorded) Jan 16 06:42:22.971: INFO: netserver-0 started at 2023-01-16 06:40:45 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:22.971: INFO: Container webserver ready: true, restart count 0 Jan 16 06:42:22.971: INFO: pod-ffb4b7bd-1f19-492a-b15d-889b768e425a started at 2023-01-16 06:42:21 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:22.971: INFO: Container test-container ready: false, restart count 0 Jan 16 06:42:22.971: INFO: coredns-559769c974-bxl2p started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:22.971: INFO: Container coredns ready: true, restart count 0 Jan 16 06:42:23.268: INFO: Latency metrics for node i-09cbbd5e7917c52c5 Jan 16 06:42:23.268: INFO: Logging node info for node i-09cdc5f20702b22f2 Jan 16 06:42:23.334: INFO: Node Info: &Node{ObjectMeta:{i-09cdc5f20702b22f2 684a930e-dce6-418d-b0e9-bf389ce1a6d1 23331 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cdc5f20702b22f2 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cdc5f20702b22f2 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.120 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-2119":"i-09cdc5f20702b22f2","ebs.csi.aws.com":"i-09cdc5f20702b22f2"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:41:20 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:42:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cdc5f20702b22f2,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:41:30 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:41:30 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:41:30 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:41:30 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.120,},NodeAddress{Type:ExternalIP,Address:52.25.16.21,},NodeAddress{Type:InternalDNS,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-16-21.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec20ae9de53dd52df737771e3cb612ee,SystemUUID:ec20ae9d-e53d-d52d-f737-771e3cb612ee,BootID:731f2d80-218c-42a6-8438-95aae6d5c60c,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0a1ba078473dea514],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0a1ba078473dea514,DevicePath:,},},Config:nil,},} Jan 16 06:42:23.335: INFO: Logging kubelet events for node i-09cdc5f20702b22f2 Jan 16 06:42:23.403: INFO: Logging pods the kubelet thinks is on node i-09cdc5f20702b22f2 Jan 16 06:42:23.485: INFO: inline-volume-tester2-5g5vv started at 2023-01-16 06:40:58 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.485: INFO: Container csi-volume-tester ready: true, restart count 0 Jan 16 06:42:23.485: INFO: netserver-1 started at 2023-01-16 06:40:45 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.485: INFO: Container webserver ready: true, restart count 0 Jan 16 06:42:23.485: INFO: startup-80516e06-d9a0-416a-b054-d59740276037 started at 2023-01-16 06:41:25 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.485: INFO: Container busybox ready: false, restart count 0 Jan 16 06:42:23.485: INFO: pod2 started at 2023-01-16 06:41:40 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.485: INFO: Container container1 ready: true, restart count 0 Jan 16 06:42:23.485: INFO: test-grpc-cc1b3bb2-e31f-4354-a994-4bab63c39b48 started at 2023-01-16 06:41:00 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.485: INFO: Container etcd ready: true, restart count 0 Jan 16 06:42:23.485: INFO: test-ss-1 started at 2023-01-16 06:41:32 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.485: INFO: Container webserver ready: true, restart count 0 Jan 16 06:42:23.485: INFO: netserver-1 started at 2023-01-16 06:42:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.485: INFO: Container webserver ready: false, restart count 0 Jan 16 06:42:23.485: INFO: cilium-wp5d7 started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded) Jan 16 06:42:23.485: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:42:23.485: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:42:23.485: INFO: hostexec-i-09cdc5f20702b22f2-rzml5 started at 2023-01-16 06:41:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.485: INFO: Container agnhost-container ready: true, restart count 0 Jan 16 06:42:23.485: INFO: test-container-pod started at 2023-01-16 06:41:14 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.485: INFO: Container webserver ready: true, restart count 0 Jan 16 06:42:23.485: INFO: csi-mockplugin-0 started at 2023-01-16 06:41:49 +0000 UTC (0+3 container statuses recorded) Jan 16 06:42:23.485: INFO: Container csi-provisioner ready: true, restart count 0 Jan 16 06:42:23.485: INFO: Container driver-registrar ready: true, restart count 0 Jan 16 06:42:23.485: INFO: Container mock ready: true, restart count 0 Jan 16 06:42:23.485: INFO: ss-0 started at 2023-01-16 06:42:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.485: INFO: Container webserver ready: true, restart count 0 Jan 16 06:42:23.485: INFO: ebs-csi-node-gk7tk started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded) Jan 16 06:42:23.485: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:42:23.485: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:42:23.485: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:42:23.785: INFO: Latency metrics for node i-09cdc5f20702b22f2 Jan 16 06:42:23.785: INFO: Logging node info for node i-0ac9adcf8852f0594 Jan 16 06:42:23.850: INFO: Node Info: &Node{ObjectMeta:{i-0ac9adcf8852f0594 5c88bf26-14b3-4940-8136-fa11c560695e 23412 0 2023-01-16 06:26:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0ac9adcf8852f0594 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-0ac9adcf8852f0594 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.221 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-449":"i-0ac9adcf8852f0594","csi-hostpath-provisioning-7349":"i-0ac9adcf8852f0594","csi-mock-csi-mock-volumes-7038":"i-0ac9adcf8852f0594","ebs.csi.aws.com":"i-0ac9adcf8852f0594"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:42:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-16 06:42:21 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0ac9adcf8852f0594,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:41:15 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:41:15 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:41:15 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:41:15 +0000 UTC,LastTransitionTime:2023-01-16 06:26:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.221,},NodeAddress{Type:ExternalIP,Address:35.90.38.176,},NodeAddress{Type:InternalDNS,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-90-38-176.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2092b4f74388fb9645e6987d21584e,SystemUUID:ec2092b4-f743-88fb-9645-e6987d21584e,BootID:6319a23e-d063-4cef-b18e-f1906415b7a3,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-7038^a0d9913f-9568-11ed-983c-3a1d1688996f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-7038^a0d9913f-9568-11ed-983c-3a1d1688996f,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-7349^ec5d3450-9568-11ed-9690-e67574852ff9,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-449^e8e315eb-9568-11ed-8015-7295eb62c406,DevicePath:,},},Config:nil,},} Jan 16 06:42:23.851: INFO: Logging kubelet events for node i-0ac9adcf8852f0594 Jan 16 06:42:23.918: INFO: Logging pods the kubelet thinks is on node i-0ac9adcf8852f0594 Jan 16 06:42:23.996: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:41:56 +0000 UTC (0+7 container statuses recorded) Jan 16 06:42:23.996: INFO: Container csi-attacher ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container csi-provisioner ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container csi-resizer ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container hostpath ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:42:23.996: INFO: ebs-csi-node-8bhxv started at 2023-01-16 06:26:07 +0000 UTC (0+3 container statuses recorded) Jan 16 06:42:23.996: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:42:23.996: INFO: csi-mockplugin-attacher-0 started at 2023-01-16 06:39:45 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.996: INFO: Container csi-attacher ready: true, restart count 0 Jan 16 06:42:23.996: INFO: pod-subpath-test-dynamicpv-vxhz started at 2023-01-16 06:42:15 +0000 UTC (1+1 container statuses recorded) Jan 16 06:42:23.996: INFO: Init container init-volume-dynamicpv-vxhz ready: false, restart count 0 Jan 16 06:42:23.996: INFO: Container test-container-subpath-dynamicpv-vxhz ready: false, restart count 0 Jan 16 06:42:23.996: INFO: pvc-volume-tester-54wpg started at 2023-01-16 06:40:12 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.996: INFO: Container volume-tester ready: true, restart count 0 Jan 16 06:42:23.996: INFO: agnhost-primary-h78v4 started at 2023-01-16 06:41:00 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.996: INFO: Container agnhost-primary ready: false, restart count 0 Jan 16 06:42:23.996: INFO: pod1 started at 2023-01-16 06:41:40 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.996: INFO: Container container1 ready: true, restart count 0 Jan 16 06:42:23.996: INFO: e2e-test-httpd-pod started at 2023-01-16 06:42:01 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.996: INFO: Container e2e-test-httpd-pod ready: true, restart count 0 Jan 16 06:42:23.996: INFO: csi-mockplugin-0 started at 2023-01-16 06:39:45 +0000 UTC (0+3 container statuses recorded) Jan 16 06:42:23.996: INFO: Container csi-provisioner ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container driver-registrar ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container mock ready: true, restart count 0 Jan 16 06:42:23.996: INFO: pod-subpath-test-dynamicpv-pd9b started at 2023-01-16 06:42:20 +0000 UTC (2+2 container statuses recorded) Jan 16 06:42:23.996: INFO: Init container init-volume-dynamicpv-pd9b ready: false, restart count 0 Jan 16 06:42:23.996: INFO: Init container test-init-subpath-dynamicpv-pd9b ready: false, restart count 0 Jan 16 06:42:23.996: INFO: Container test-container-subpath-dynamicpv-pd9b ready: false, restart count 0 Jan 16 06:42:23.996: INFO: Container test-container-volume-dynamicpv-pd9b ready: false, restart count 0 Jan 16 06:42:23.996: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:41:49 +0000 UTC (0+7 container statuses recorded) Jan 16 06:42:23.996: INFO: Container csi-attacher ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container csi-provisioner ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container csi-resizer ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container hostpath ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:42:23.996: INFO: netserver-2 started at 2023-01-16 06:42:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.996: INFO: Container webserver ready: true, restart count 0 Jan 16 06:42:23.996: INFO: bin-falsece17fdb5-41ac-4a72-b3cf-b477ede2a97c started at 2023-01-16 06:42:06 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.996: INFO: Container bin-falsece17fdb5-41ac-4a72-b3cf-b477ede2a97c ready: false, restart count 0 Jan 16 06:42:23.996: INFO: netserver-2 started at 2023-01-16 06:40:45 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.996: INFO: Container webserver ready: true, restart count 0 Jan 16 06:42:23.996: INFO: busybox-privileged-false-6e9090f2-5527-4aa3-ba32-023536f95b57 started at 2023-01-16 06:41:16 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.996: INFO: Container busybox-privileged-false-6e9090f2-5527-4aa3-ba32-023536f95b57 ready: false, restart count 0 Jan 16 06:42:23.996: INFO: httpd started at 2023-01-16 06:41:43 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:23.996: INFO: Container httpd ready: true, restart count 0 Jan 16 06:42:23.996: INFO: cilium-9tdp4 started at 2023-01-16 06:26:07 +0000 UTC (1+1 container statuses recorded) Jan 16 06:42:23.996: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:42:23.996: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:42:24.314: INFO: Latency metrics for node i-0ac9adcf8852f0594 Jan 16 06:42:24.314: INFO: Logging node info for node i-0b3e7c1b3ffabe4e1 Jan 16 06:42:24.379: INFO: Node Info: &Node{ObjectMeta:{i-0b3e7c1b3ffabe4e1 f1f7f96e-8f6d-4178-9a45-c16cedbd8510 23543 0 2023-01-16 06:25:59 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b3e7c1b3ffabe4e1 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-0b3e7c1b3ffabe4e1 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.59.231 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-2403":"i-0b3e7c1b3ffabe4e1","ebs.csi.aws.com":"i-0b3e7c1b3ffabe4e1"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2023-01-16 06:42:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0b3e7c1b3ffabe4e1,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:41:48 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:41:48 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:41:48 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:41:48 +0000 UTC,LastTransitionTime:2023-01-16 06:26:13 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.231,},NodeAddress{Type:ExternalIP,Address:52.12.10.229,},NodeAddress{Type:InternalDNS,Address:i-0b3e7c1b3ffabe4e1.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b3e7c1b3ffabe4e1.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-12-10-229.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec26e5d5e832a48110ff0427b389577a,SystemUUID:ec26e5d5-e832-a481-10ff-0427b389577a,BootID:21853215-8129-4e64-b14a-211cf0ee9b99,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 16 06:42:24.380: INFO: Logging kubelet events for node i-0b3e7c1b3ffabe4e1 Jan 16 06:42:24.448: INFO: Logging pods the kubelet thinks is on node i-0b3e7c1b3ffabe4e1 Jan 16 06:42:24.527: INFO: hostexec-i-0b3e7c1b3ffabe4e1-6sj57 started at 2023-01-16 06:41:56 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:24.527: INFO: Container agnhost-container ready: true, restart count 0 Jan 16 06:42:24.527: INFO: inline-volume-cwlk4 started at 2023-01-16 06:40:29 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:24.527: INFO: Container volume-tester ready: false, restart count 0 Jan 16 06:42:24.527: INFO: ebs-csi-node-zg8kr started at 2023-01-16 06:26:00 +0000 UTC (0+3 container statuses recorded) Jan 16 06:42:24.527: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:42:24.527: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:42:24.527: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:42:24.527: INFO: hostexec-i-0b3e7c1b3ffabe4e1-k8wcd started at 2023-01-16 06:42:08 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:24.527: INFO: Container agnhost-container ready: true, restart count 0 Jan 16 06:42:24.527: INFO: hostexec-i-0b3e7c1b3ffabe4e1-9krr6 started at 2023-01-16 06:42:21 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:24.527: INFO: Container agnhost-container ready: true, restart count 0 Jan 16 06:42:24.527: INFO: pod-8cbd9b35-c6ea-4332-91b1-90bf6b17b75b started at 2023-01-16 06:42:00 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:24.527: INFO: Container write-pod ready: true, restart count 0 Jan 16 06:42:24.527: INFO: pod-subpath-test-preprovisionedpv-2qgf started at 2023-01-16 06:42:20 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:24.527: INFO: Container test-container-subpath-preprovisionedpv-2qgf ready: false, restart count 0 Jan 16 06:42:24.527: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:42:21 +0000 UTC (0+7 container statuses recorded) Jan 16 06:42:24.527: INFO: Container csi-attacher ready: true, restart count 0 Jan 16 06:42:24.527: INFO: Container csi-provisioner ready: true, restart count 0 Jan 16 06:42:24.527: INFO: Container csi-resizer ready: true, restart count 0 Jan 16 06:42:24.527: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 16 06:42:24.527: INFO: Container hostpath ready: true, restart count 0 Jan 16 06:42:24.527: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:42:24.527: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:42:24.527: INFO: netserver-3 started at 2023-01-16 06:42:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:24.527: INFO: Container webserver ready: false, restart count 0 Jan 16 06:42:24.527: INFO: coredns-559769c974-v2nv5 started at 2023-01-16 06:26:26 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:24.527: INFO: Container coredns ready: true, restart count 0 Jan 16 06:42:24.527: INFO: test-ss-0 started at 2023-01-16 06:40:32 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:24.527: INFO: Container webserver ready: true, restart count 0 Jan 16 06:42:24.527: INFO: test-webserver-0efc1bd4-0fc8-45dd-8b1a-cf0f438d860b started at 2023-01-16 06:41:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:24.527: INFO: Container test-webserver ready: true, restart count 0 Jan 16 06:42:24.527: INFO: netserver-3 started at 2023-01-16 06:40:45 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:24.527: INFO: Container webserver ready: true, restart count 0 Jan 16 06:42:24.527: INFO: cilium-l9bsn started at 2023-01-16 06:26:00 +0000 UTC (1+1 container statuses recorded) Jan 16 06:42:24.527: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:42:24.527: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:42:24.527: INFO: agnhost-primary-n4tx6 started at 2023-01-16 06:40:59 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:24.527: INFO: Container agnhost-primary ready: false, restart count 0 Jan 16 06:42:24.874: INFO: Latency metrics for node i-0b3e7c1b3ffabe4e1 Jan 16 06:42:24.874: INFO: Logging node info for node i-0bcdb98d646a60a68 Jan 16 06:42:24.939: INFO: Node Info: &Node{ObjectMeta:{i-0bcdb98d646a60a68 ef8446c9-df03-44e4-bfed-57c522916505 21802 0 2023-01-16 06:24:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0bcdb98d646a60a68 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.49.7 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bcdb98d646a60a68"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-16 06:24:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-16 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:24:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:24:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:24:54 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-01-16 06:41:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0bcdb98d646a60a68,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:41:27 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:41:27 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:41:27 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:41:27 +0000 UTC,LastTransitionTime:2023-01-16 06:24:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.49.7,},NodeAddress{Type:ExternalIP,Address:35.164.29.62,},NodeAddress{Type:InternalDNS,Address:i-0bcdb98d646a60a68.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0bcdb98d646a60a68.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-164-29-62.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2e02b7de327b42fbf146d4a7087a79,SystemUUID:ec2e02b7-de32-7b42-fbf1-46d4a7087a79,BootID:8ddb4053-2513-4e18-b49d-f322ee6d1f6e,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43640493,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42802003,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967313,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 16 06:42:24.939: INFO: Logging kubelet events for node i-0bcdb98d646a60a68 Jan 16 06:42:25.006: INFO: Logging pods the kubelet thinks is on node i-0bcdb98d646a60a68 Jan 16 06:42:25.077: INFO: etcd-manager-events-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:25.077: INFO: Container etcd-manager ready: true, restart count 0 Jan 16 06:42:25.077: INFO: kube-controller-manager-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:25.077: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 16 06:42:25.077: INFO: kube-scheduler-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:25.077: INFO: Container kube-scheduler ready: true, restart count 0 Jan 16 06:42:25.077: INFO: ebs-csi-node-9sg8r started at 2023-01-16 06:24:42 +0000 UTC (0+3 container statuses recorded) Jan 16 06:42:25.077: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:42:25.077: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:42:25.077: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:42:25.077: INFO: dns-controller-69987775c6-8t6m5 started at 2023-01-16 06:24:43 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:25.077: INFO: Container dns-controller ready: true, restart count 0 Jan 16 06:42:25.077: INFO: ebs-csi-controller-5bd98b456f-54tj7 started at 2023-01-16 06:24:43 +0000 UTC (0+5 container statuses recorded) Jan 16 06:42:25.077: INFO: Container csi-attacher ready: true, restart count 0 Jan 16 06:42:25.077: INFO: Container csi-provisioner ready: true, restart count 0 Jan 16 06:42:25.077: INFO: Container csi-resizer ready: true, restart count 0 Jan 16 06:42:25.077: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:42:25.077: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:42:25.077: INFO: etcd-manager-main-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:25.077: INFO: Container etcd-manager ready: true, restart count 0 Jan 16 06:42:25.077: INFO: kube-apiserver-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+2 container statuses recorded) Jan 16 06:42:25.077: INFO: Container healthcheck ready: true, restart count 0 Jan 16 06:42:25.077: INFO: Container kube-apiserver ready: true, restart count 1 Jan 16 06:42:25.077: INFO: cilium-qwpgj started at 2023-01-16 06:24:42 +0000 UTC (1+1 container statuses recorded) Jan 16 06:42:25.077: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:42:25.077: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:42:25.077: INFO: kops-controller-bjgn2 started at 2023-01-16 06:24:42 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:25.077: INFO: Container kops-controller ready: true, restart count 0 Jan 16 06:42:25.077: INFO: aws-cloud-controller-manager-fb8z2 started at 2023-01-16 06:24:42 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:25.077: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 16 06:42:25.077: INFO: cilium-operator-5dd44dc49f-sftdz started at 2023-01-16 06:24:43 +0000 UTC (0+1 container statuses recorded) Jan 16 06:42:25.077: INFO: Container cilium-operator ready: true, restart count 0 Jan 16 06:42:25.352: INFO: Latency metrics for node i-0bcdb98d646a60a68 [DeferCleanup (Each)] [sig-cli] Kubectl client tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "kubectl-4697" for this suite. �[38;5;243m01/16/23 06:42:25.352�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sConntrack\sshould\sbe\sable\sto\spreserve\sUDP\straffic\swhen\sinitial\sunready\sendpoints\sget\sready$'
test/e2e/framework/pod/pod_client.go:107 k8s.io/kubernetes/test/e2e/framework/pod.(*PodClient).CreateSync(0xc00106c618, 0x6d?) test/e2e/framework/pod/pod_client.go:107 +0x94 k8s.io/kubernetes/test/e2e/network.glob..func1.5() test/e2e/network/conntrack.go:316 +0x3a9from junit_01.xml
[BeforeEach] [sig-network] Conntrack set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/16/23 06:33:05.813�[0m Jan 16 06:33:05.813: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename conntrack �[38;5;243m01/16/23 06:33:05.814�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/16/23 06:33:06.002�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/16/23 06:33:06.126�[0m [BeforeEach] [sig-network] Conntrack test/e2e/framework/metrics/init/init.go:31 [BeforeEach] [sig-network] Conntrack test/e2e/network/conntrack.go:98 [It] should be able to preserve UDP traffic when initial unready endpoints get ready test/e2e/network/conntrack.go:295 �[1mSTEP:�[0m creating a UDP service svc-udp with type=ClusterIP in conntrack-2130 �[38;5;243m01/16/23 06:33:06.314�[0m �[1mSTEP:�[0m creating a client pod for probing the service svc-udp �[38;5;243m01/16/23 06:33:06.388�[0m Jan 16 06:33:06.465: INFO: Waiting up to 5m0s for pod "pod-client" in namespace "conntrack-2130" to be "running and ready" Jan 16 06:33:06.527: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 62.104812ms Jan 16 06:33:06.527: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:08.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2.125795412s Jan 16 06:33:08.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:10.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4.124840036s Jan 16 06:33:10.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:12.592: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 6.126861015s Jan 16 06:33:12.592: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:14.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 8.124780513s Jan 16 06:33:14.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:16.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 10.125814367s Jan 16 06:33:16.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:18.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 12.125783001s Jan 16 06:33:18.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:20.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 14.124661113s Jan 16 06:33:20.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:22.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 16.125397372s Jan 16 06:33:22.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:24.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 18.124920139s Jan 16 06:33:24.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:26.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 20.12558062s Jan 16 06:33:26.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:28.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 22.124695478s Jan 16 06:33:28.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:30.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 24.125264611s Jan 16 06:33:30.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:32.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 26.124610468s Jan 16 06:33:32.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:34.594: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 28.129039756s Jan 16 06:33:34.594: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:36.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 30.125457922s Jan 16 06:33:36.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:38.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 32.125555757s Jan 16 06:33:38.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:40.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 34.125534761s Jan 16 06:33:40.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:42.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 36.124961096s Jan 16 06:33:42.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:44.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 38.126180234s Jan 16 06:33:44.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:46.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 40.124896857s Jan 16 06:33:46.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:48.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 42.124549182s Jan 16 06:33:48.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:50.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 44.125260867s Jan 16 06:33:50.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:52.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 46.125692881s Jan 16 06:33:52.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:54.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 48.124891087s Jan 16 06:33:54.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:56.593: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 50.128074044s Jan 16 06:33:56.593: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:33:58.603: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 52.137693696s Jan 16 06:33:58.603: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:00.593: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 54.127530374s Jan 16 06:34:00.593: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:02.592: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 56.126693522s Jan 16 06:34:02.592: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:04.958: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 58.493270469s Jan 16 06:34:04.958: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:06.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.125367502s Jan 16 06:34:06.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:08.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.125241792s Jan 16 06:34:08.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:10.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.125199763s Jan 16 06:34:10.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:12.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.125525425s Jan 16 06:34:12.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:14.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.125553821s Jan 16 06:34:14.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:16.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.12544232s Jan 16 06:34:16.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:18.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.125553883s Jan 16 06:34:18.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:20.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.12457299s Jan 16 06:34:20.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:22.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.125109263s Jan 16 06:34:22.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:24.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.125299201s Jan 16 06:34:24.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:26.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.12551595s Jan 16 06:34:26.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:28.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.125710539s Jan 16 06:34:28.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:30.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.12495266s Jan 16 06:34:30.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:32.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.125346335s Jan 16 06:34:32.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:34.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.124776341s Jan 16 06:34:34.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:36.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.125584864s Jan 16 06:34:36.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:38.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.125429562s Jan 16 06:34:38.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:40.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.125296169s Jan 16 06:34:40.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:42.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.124807102s Jan 16 06:34:42.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:44.592: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.126585252s Jan 16 06:34:44.592: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:46.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.124797011s Jan 16 06:34:46.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:48.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.124669962s Jan 16 06:34:48.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:50.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.125107534s Jan 16 06:34:50.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:52.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.124791166s Jan 16 06:34:52.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:54.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.12529459s Jan 16 06:34:54.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:56.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.125250968s Jan 16 06:34:56.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:34:58.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.125858199s Jan 16 06:34:58.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:00.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.125668302s Jan 16 06:35:00.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:02.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.125775679s Jan 16 06:35:02.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:04.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.124525171s Jan 16 06:35:04.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:06.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.125104387s Jan 16 06:35:06.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:08.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.124855676s Jan 16 06:35:08.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:10.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.125252798s Jan 16 06:35:10.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:12.593: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.128165822s Jan 16 06:35:12.593: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:14.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.124949798s Jan 16 06:35:14.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:16.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.125569275s Jan 16 06:35:16.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:18.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.125566808s Jan 16 06:35:18.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:20.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.124963049s Jan 16 06:35:20.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:22.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.124996094s Jan 16 06:35:22.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:24.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.124879547s Jan 16 06:35:24.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:26.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.125637387s Jan 16 06:35:26.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:28.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.125040756s Jan 16 06:35:28.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:30.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.125388166s Jan 16 06:35:30.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:32.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.125478987s Jan 16 06:35:32.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:34.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.125495753s Jan 16 06:35:34.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:36.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.124875877s Jan 16 06:35:36.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:38.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.125049528s Jan 16 06:35:38.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:40.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.124834119s Jan 16 06:35:40.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:42.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.125610146s Jan 16 06:35:42.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:44.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.12502393s Jan 16 06:35:44.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:46.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.125472023s Jan 16 06:35:46.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:48.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.124885831s Jan 16 06:35:48.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:50.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.125438385s Jan 16 06:35:50.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:52.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.125401871s Jan 16 06:35:52.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:54.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.124757871s Jan 16 06:35:54.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:56.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.125037385s Jan 16 06:35:56.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:35:58.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.125279316s Jan 16 06:35:58.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:00.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.124861515s Jan 16 06:36:00.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:02.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.124663272s Jan 16 06:36:02.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:04.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.124564145s Jan 16 06:36:04.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:06.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.12560835s Jan 16 06:36:06.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:08.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.124529779s Jan 16 06:36:08.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:10.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.125374382s Jan 16 06:36:10.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:12.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.125466619s Jan 16 06:36:12.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:14.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.124539867s Jan 16 06:36:14.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:16.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.124714189s Jan 16 06:36:16.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:18.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.125536213s Jan 16 06:36:18.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:20.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.125390715s Jan 16 06:36:20.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:22.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.125261573s Jan 16 06:36:22.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:24.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.125280694s Jan 16 06:36:24.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:26.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.124628943s Jan 16 06:36:26.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:28.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.124753447s Jan 16 06:36:28.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:30.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.125182808s Jan 16 06:36:30.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:32.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.125726101s Jan 16 06:36:32.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:34.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.124928616s Jan 16 06:36:34.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:36.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.12533236s Jan 16 06:36:36.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:38.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.125675872s Jan 16 06:36:38.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:40.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.125128838s Jan 16 06:36:40.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:42.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.125565277s Jan 16 06:36:42.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:44.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.124745352s Jan 16 06:36:44.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:46.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.124619459s Jan 16 06:36:46.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:48.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.124969292s Jan 16 06:36:48.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:50.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.124905855s Jan 16 06:36:50.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:52.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.125565015s Jan 16 06:36:52.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:54.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.125497861s Jan 16 06:36:54.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:56.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.1246434s Jan 16 06:36:56.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:36:58.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.12554689s Jan 16 06:36:58.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:00.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.125271789s Jan 16 06:37:00.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:02.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.12467111s Jan 16 06:37:02.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:04.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.125682646s Jan 16 06:37:04.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:06.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.125352306s Jan 16 06:37:06.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:08.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.125006533s Jan 16 06:37:08.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:10.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.125051006s Jan 16 06:37:10.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:12.589: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.124442985s Jan 16 06:37:12.589: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:14.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.125764235s Jan 16 06:37:14.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:16.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.124800627s Jan 16 06:37:16.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:18.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.124916773s Jan 16 06:37:18.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:20.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.124802329s Jan 16 06:37:20.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:22.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.124949404s Jan 16 06:37:22.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:24.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.124763578s Jan 16 06:37:24.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:26.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.125412493s Jan 16 06:37:26.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:28.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.125333528s Jan 16 06:37:28.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:30.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.125542113s Jan 16 06:37:30.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:32.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.124859406s Jan 16 06:37:32.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:34.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.124942097s Jan 16 06:37:34.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:36.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.125346566s Jan 16 06:37:36.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:38.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.124714319s Jan 16 06:37:38.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:40.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.124868092s Jan 16 06:37:40.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:42.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.124680115s Jan 16 06:37:42.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:44.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.125676005s Jan 16 06:37:44.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:46.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.125470431s Jan 16 06:37:46.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:48.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.124706196s Jan 16 06:37:48.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:50.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.12531012s Jan 16 06:37:50.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:52.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.12554396s Jan 16 06:37:52.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:54.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.124704681s Jan 16 06:37:54.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:56.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.124749076s Jan 16 06:37:56.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:37:58.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.125015802s Jan 16 06:37:58.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:38:00.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.124705795s Jan 16 06:38:00.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:38:02.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.124747484s Jan 16 06:38:02.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:38:04.591: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.125592485s Jan 16 06:38:04.591: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:38:06.590: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.125455842s Jan 16 06:38:06.590: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:38:06.653: INFO: Pod "pod-client": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.187575886s Jan 16 06:38:06.653: INFO: The phase of Pod pod-client is Pending, waiting for it to be Running (with Ready = true) Jan 16 06:38:06.654: INFO: Unexpected error: <*pod.timeoutError | 0xc004d803f0>: { msg: "timed out while waiting for pod conntrack-2130/pod-client to be running and ready", observedObjects: [ <*v1.Pod | 0xc003f08d80>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "pod-client", GenerateName: "", Namespace: "conntrack-2130", SelfLink: "", UID: "037f38b4-1b38-444d-b8fb-704432c2cac1", ResourceVersion: "13410", Generation: 0, CreationTimestamp: { Time: { wall: 0, ext: 63809447586, loc: { name: "Local", zone: [ {name: "UTC", offset: 0, isDST: false}, ], tx: [ { when: -576460752303423488, index: 0, isstd: false, isutc: false, }, ], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: "UTC", offset: 0, isDST: false}, }, }, }, DeletionTimestamp: nil, DeletionGracePeriodSeconds: nil, Labels: nil, Annotations: nil, OwnerReferences: nil, Finalizers: nil, ManagedFields: [ { Manager: "e2e.test", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63809447586, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:spec\":{\"f:affinity\":{\".\":{},\"f:nodeAffinity\":{\".\":{},\"f:requiredDuringSchedulingIgnoredDuringExecution\":{}}},\"f:containers\":{\"k:{\\\"name\\\":\\\"pod-client\\\"}\":{\".\":{},\"f:args\":{},\"f:command\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:securityContext\":{},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:terminationGracePeriodSeconds\":{}}}", }, Subresource: "", }, { Manager: "kubelet", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63809447588, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}", }, Subresource: "status", }, ], }, Spec: { Volumes: [ { Name: "kube-api-access-njc2g", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: nil, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: { Sources: [ { Secret: ..., DownwardAPI: ..., ConfigMap: ..., ServiceAccountToken: ..., }, { Secret: ..., DownwardAPI: ..., ConfigMap: ..., ServiceAccountToken: ..., }, { Secret: ..., DownwardAPI: ..., ConfigMap: ..., ServiceAccountToken: ..., }, ], DefaultMode: 420, }, ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Jan 16 06:38:06.654: FAIL: timed out while waiting for pod conntrack-2130/pod-client to be running and ready Full Stack Trace k8s.io/kubernetes/test/e2e/framework/pod.(*PodClient).CreateSync(0xc00106c618, 0x6d?) test/e2e/framework/pod/pod_client.go:107 +0x94 k8s.io/kubernetes/test/e2e/network.glob..func1.5() test/e2e/network/conntrack.go:316 +0x3a9 [AfterEach] [sig-network] Conntrack test/e2e/framework/node/init/init.go:32 Jan 16 06:38:06.654: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-network] Conntrack test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-network] Conntrack dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/16/23 06:38:06.717�[0m �[1mSTEP:�[0m Collecting events from namespace "conntrack-2130". �[38;5;243m01/16/23 06:38:06.717�[0m �[1mSTEP:�[0m Found 11 events. �[38;5;243m01/16/23 06:38:06.779�[0m Jan 16 06:38:06.780: INFO: At 2023-01-16 06:33:06 +0000 UTC - event for pod-client: {default-scheduler } Scheduled: Successfully assigned conntrack-2130/pod-client to i-09cdc5f20702b22f2 Jan 16 06:38:06.780: INFO: At 2023-01-16 06:33:07 +0000 UTC - event for pod-client: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "46864d844f74ca511d9e0ea9102b57d90c41c9c749f6fbd519185d6204a84aa1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:06.780: INFO: At 2023-01-16 06:33:18 +0000 UTC - event for pod-client: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "fa3ef2413bedf04f6eb06756edfd9ff7c28355e4ac8af30cf655cf3e6e65b3e2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:06.780: INFO: At 2023-01-16 06:33:31 +0000 UTC - event for pod-client: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0ab84089e77a41d274d45ef5491978fa7d05a58bca973bb8802deb1fb56212f6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:06.780: INFO: At 2023-01-16 06:33:46 +0000 UTC - event for pod-client: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3840b2a09a5f5bfbb0523c20f3ed501bf84a1be2dbb036531453dd3341231325": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:06.780: INFO: At 2023-01-16 06:34:01 +0000 UTC - event for pod-client: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0f51a9522584553798ce67e8daf682d64c58ac476d54055a53980f148e2fd52e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:06.780: INFO: At 2023-01-16 06:34:15 +0000 UTC - event for pod-client: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9392d2aff4861e3f6a13c4e2fc2f6cec235a39850ab38155e7df8db73f63700d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:06.780: INFO: At 2023-01-16 06:34:30 +0000 UTC - event for pod-client: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bc7a0041f0e2349c61b5bc28b13292c7a2938b9259c459d4269b6c3f1e47298d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:06.780: INFO: At 2023-01-16 06:34:44 +0000 UTC - event for pod-client: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "41cba11a1a9d87e057ed1875fc5ca120d4b83f9b6ca8dd56afa642aa44598df9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:06.780: INFO: At 2023-01-16 06:34:57 +0000 UTC - event for pod-client: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "12a10f8d087e6247bcffb675a76dd536c6ee5b700f99241f0d031ac42eec7d85": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:06.780: INFO: At 2023-01-16 06:35:08 +0000 UTC - event for pod-client: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b4d5499ff3736a055e47b7c5fbe1d6e9d8af8110d6409d14a5071f3ba94fde32": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:38:06.842: INFO: POD NODE PHASE GRACE CONDITIONS Jan 16 06:38:06.842: INFO: pod-client i-09cdc5f20702b22f2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:33:06 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:33:06 +0000 UTC ContainersNotReady containers with unready status: [pod-client]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:33:06 +0000 UTC ContainersNotReady containers with unready status: [pod-client]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:33:06 +0000 UTC }] Jan 16 06:38:06.842: INFO: Jan 16 06:38:06.905: INFO: Unable to fetch conntrack-2130/pod-client/pod-client logs: the server rejected our request for an unknown reason (get pods pod-client) Jan 16 06:38:06.968: INFO: Logging node info for node i-09cbbd5e7917c52c5 Jan 16 06:38:07.030: INFO: Node Info: &Node{ObjectMeta:{i-09cbbd5e7917c52c5 516d50c9-4cfe-4b03-9036-fb7eaefbf36d 15320 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cbbd5e7917c52c5 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cbbd5e7917c52c5 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.43.77 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-3020":"i-09cbbd5e7917c52c5","ebs.csi.aws.com":"i-09cbbd5e7917c52c5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:32:44 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:37:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cbbd5e7917c52c5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:26:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.77,},NodeAddress{Type:ExternalIP,Address:52.25.36.68,},NodeAddress{Type:InternalDNS,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-36-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2589b28acc26043d6f13727e02d530,SystemUUID:ec2589b2-8acc-2604-3d6f-13727e02d530,BootID:f8305071-d74c-44c0-a267-8cc84ea9ad7b,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-3020^9489448c-9567-11ed-b056-f2aad9c35bb6],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-3020^9489448c-9567-11ed-b056-f2aad9c35bb6,DevicePath:,},},Config:nil,},} Jan 16 06:38:07.031: INFO: Logging kubelet events for node i-09cbbd5e7917c52c5 Jan 16 06:38:07.096: INFO: Logging pods the kubelet thinks is on node i-09cbbd5e7917c52c5 Jan 16 06:38:07.168: INFO: csi-mockplugin-0 started at 2023-01-16 06:32:16 +0000 UTC (0+4 container statuses recorded) Jan 16 06:38:07.168: INFO: Container busybox ready: false, restart count 0 Jan 16 06:38:07.168: INFO: Container csi-provisioner ready: false, restart count 0 Jan 16 06:38:07.168: INFO: Container driver-registrar ready: false, restart count 0 Jan 16 06:38:07.168: INFO: Container mock ready: false, restart count 0 Jan 16 06:38:07.168: INFO: cilium-xxhgc started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:38:07.168: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:38:07.168: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-vvtkb started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.168: INFO: hostpath-symlink-prep-provisioning-9295 started at 2023-01-16 06:34:07 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container init-volume-provisioning-9295 ready: false, restart count 0 Jan 16 06:38:07.168: INFO: inline-volume-tester-5b6t4 started at 2023-01-16 06:32:43 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 16 06:38:07.168: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-blsx2 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.168: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-pwdrm started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.168: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c9wj5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.168: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gk8f6 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.168: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jmpsk started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.168: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-zrtfg started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.168: INFO: ebs-csi-node-smxsx started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded) Jan 16 06:38:07.168: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:38:07.168: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:38:07.168: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:38:07.168: INFO: coredns-autoscaler-7cb5c5b969-lt7sb started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container autoscaler ready: true, restart count 0 Jan 16 06:38:07.168: INFO: coredns-559769c974-bxl2p started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container coredns ready: true, restart count 0 Jan 16 06:38:07.168: INFO: csi-mockplugin-0 started at 2023-01-16 06:32:45 +0000 UTC (0+3 container statuses recorded) Jan 16 06:38:07.168: INFO: Container csi-provisioner ready: false, restart count 0 Jan 16 06:38:07.168: INFO: Container driver-registrar ready: false, restart count 0 Jan 16 06:38:07.168: INFO: Container mock ready: false, restart count 0 Jan 16 06:38:07.168: INFO: csi-mockplugin-attacher-0 started at 2023-01-16 06:32:45 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container csi-attacher ready: false, restart count 0 Jan 16 06:38:07.168: INFO: rs-qcmvk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container donothing ready: false, restart count 0 Jan 16 06:38:07.168: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-m6w4z started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.168: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0 Jan 16 06:38:07.168: INFO: netserver-0 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container webserver ready: false, restart count 0 Jan 16 06:38:07.168: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:36:13 +0000 UTC (0+7 container statuses recorded) Jan 16 06:38:07.168: INFO: Container csi-attacher ready: false, restart count 0 Jan 16 06:38:07.168: INFO: Container csi-provisioner ready: false, restart count 0 Jan 16 06:38:07.168: INFO: Container csi-resizer ready: false, restart count 0 Jan 16 06:38:07.168: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 16 06:38:07.168: INFO: Container hostpath ready: false, restart count 0 Jan 16 06:38:07.168: INFO: Container liveness-probe ready: false, restart count 0 Jan 16 06:38:07.168: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 16 06:38:07.168: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:32:40 +0000 UTC (0+7 container statuses recorded) Jan 16 06:38:07.168: INFO: Container csi-attacher ready: true, restart count 0 Jan 16 06:38:07.168: INFO: Container csi-provisioner ready: true, restart count 0 Jan 16 06:38:07.168: INFO: Container csi-resizer ready: true, restart count 0 Jan 16 06:38:07.168: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 16 06:38:07.168: INFO: Container hostpath ready: true, restart count 0 Jan 16 06:38:07.168: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:38:07.168: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:38:07.168: INFO: rs-9ttnx started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container donothing ready: false, restart count 0 Jan 16 06:38:07.168: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nwpjj started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.168: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.440: INFO: Latency metrics for node i-09cbbd5e7917c52c5 Jan 16 06:38:07.440: INFO: Logging node info for node i-09cdc5f20702b22f2 Jan 16 06:38:07.502: INFO: Node Info: &Node{ObjectMeta:{i-09cdc5f20702b22f2 684a930e-dce6-418d-b0e9-bf389ce1a6d1 15164 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cdc5f20702b22f2 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cdc5f20702b22f2 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.120 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cdc5f20702b22f2"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-16 06:37:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cdc5f20702b22f2,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.120,},NodeAddress{Type:ExternalIP,Address:52.25.16.21,},NodeAddress{Type:InternalDNS,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-16-21.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec20ae9de53dd52df737771e3cb612ee,SystemUUID:ec20ae9d-e53d-d52d-f737-771e3cb612ee,BootID:731f2d80-218c-42a6-8438-95aae6d5c60c,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 16 06:38:07.503: INFO: Logging kubelet events for node i-09cdc5f20702b22f2 Jan 16 06:38:07.567: INFO: Logging pods the kubelet thinks is on node i-09cdc5f20702b22f2 Jan 16 06:38:07.638: INFO: all-succeed-n4z2d started at 2023-01-16 06:34:38 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container c ready: false, restart count 0 Jan 16 06:38:07.638: INFO: rs-99bkg started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container donothing ready: true, restart count 0 Jan 16 06:38:07.638: INFO: successful-jobs-history-limit-27897515-25xjd started at 2023-01-16 06:35:00 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container c ready: false, restart count 0 Jan 16 06:38:07.638: INFO: netserver-1 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container webserver ready: false, restart count 0 Jan 16 06:38:07.638: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5d2hr started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.638: INFO: cilium-wp5d7 started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:38:07.638: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:38:07.638: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ptdxh started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.638: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jh4ht started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.638: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s5kgk started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.638: INFO: all-pods-removed-6lcv7 started at 2023-01-16 06:33:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container c ready: true, restart count 0 Jan 16 06:38:07.638: INFO: hostexec-i-09cdc5f20702b22f2-6w69n started at 2023-01-16 06:34:18 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container agnhost-container ready: true, restart count 0 Jan 16 06:38:07.638: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7tft7 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.638: INFO: rs-h9gkk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container donothing ready: false, restart count 0 Jan 16 06:38:07.638: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.638: INFO: all-succeed-wcs5x started at 2023-01-16 06:34:52 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container c ready: false, restart count 0 Jan 16 06:38:07.638: INFO: pod-771554b8-0ed1-4422-82d5-a0a4dc92ec52 started at 2023-01-16 06:34:29 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container write-pod ready: false, restart count 0 Jan 16 06:38:07.638: INFO: pod-secrets-5bc87a87-80c1-4cdc-bedc-1ae4930ee9bc started at 2023-01-16 06:35:05 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container secret-volume-test ready: false, restart count 0 Jan 16 06:38:07.638: INFO: service-deletion-9hpkv started at 2023-01-16 06:35:33 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container service-deletion ready: false, restart count 0 Jan 16 06:38:07.638: INFO: ebs-csi-node-gk7tk started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded) Jan 16 06:38:07.638: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:38:07.638: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:38:07.638: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:38:07.638: INFO: successful-jobs-history-limit-27897516-zff4c started at 2023-01-16 06:36:00 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container c ready: false, restart count 0 Jan 16 06:38:07.638: INFO: rs-qzzlk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container donothing ready: false, restart count 0 Jan 16 06:38:07.638: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.638: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-x7cbm started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.638: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-z5z6l started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.638: INFO: pod-client started at 2023-01-16 06:33:06 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container pod-client ready: false, restart count 0 Jan 16 06:38:07.638: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2jtsn started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:07.638: INFO: successful-jobs-history-limit-27897513-n4m26 started at 2023-01-16 06:33:00 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container c ready: false, restart count 0 Jan 16 06:38:07.638: INFO: successful-jobs-history-limit-27897514-pt4qd started at 2023-01-16 06:34:00 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container c ready: false, restart count 0 Jan 16 06:38:07.638: INFO: all-pods-removed-hgfff started at 2023-01-16 06:33:12 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container c ready: false, restart count 0 Jan 16 06:38:07.638: INFO: all-succeed-pff5k started at 2023-01-16 06:33:56 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container c ready: false, restart count 0 Jan 16 06:38:07.638: INFO: hairpin started at 2023-01-16 06:34:50 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:07.638: INFO: Container agnhost-container ready: false, restart count 0 Jan 16 06:38:07.915: INFO: Latency metrics for node i-09cdc5f20702b22f2 Jan 16 06:38:07.915: INFO: Logging node info for node i-0ac9adcf8852f0594 Jan 16 06:38:07.977: INFO: Node Info: &Node{ObjectMeta:{i-0ac9adcf8852f0594 5c88bf26-14b3-4940-8136-fa11c560695e 15325 0 2023-01-16 06:26:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0ac9adcf8852f0594 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-0ac9adcf8852f0594 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.221 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-4883":"i-0ac9adcf8852f0594","ebs.csi.aws.com":"i-0ac9adcf8852f0594"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-16 06:34:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-16 06:37:57 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0ac9adcf8852f0594,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.221,},NodeAddress{Type:ExternalIP,Address:35.90.38.176,},NodeAddress{Type:InternalDNS,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-90-38-176.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2092b4f74388fb9645e6987d21584e,SystemUUID:ec2092b4-f743-88fb-9645-e6987d21584e,BootID:6319a23e-d063-4cef-b18e-f1906415b7a3,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-051cd447c54259a6e kubernetes.io/csi/ebs.csi.aws.com^vol-0c3cdffdafd49e4d9],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-051cd447c54259a6e,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0c3cdffdafd49e4d9,DevicePath:,},},Config:nil,},} Jan 16 06:38:07.977: INFO: Logging kubelet events for node i-0ac9adcf8852f0594 Jan 16 06:38:08.042: INFO: Logging pods the kubelet thinks is on node i-0ac9adcf8852f0594 Jan 16 06:38:08.113: INFO: pvc-volume-tester-48722 started at 2023-01-16 06:35:01 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.113: INFO: Container volume-tester ready: false, restart count 0 Jan 16 06:38:08.113: INFO: pod-8becfd69-702e-46ad-bf8e-2de0749dc180 started at 2023-01-16 06:33:46 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.113: INFO: Container test-container ready: false, restart count 0 Jan 16 06:38:08.113: INFO: liveness-470354a2-716e-4341-b41b-afc9c1378c49 started at 2023-01-16 06:35:01 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.113: INFO: Container agnhost-container ready: false, restart count 0 Jan 16 06:38:08.113: INFO: rs-vwwhw started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.113: INFO: Container donothing ready: false, restart count 0 Jan 16 06:38:08.113: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-b9xdt started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.113: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.113: INFO: successful-jobs-history-limit-27897517-7lln2 started at 2023-01-16 06:37:00 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.113: INFO: Container c ready: false, restart count 0 Jan 16 06:38:08.113: INFO: rs-7fgjp started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.113: INFO: Container donothing ready: true, restart count 0 Jan 16 06:38:08.113: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.113: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0 Jan 16 06:38:08.114: INFO: successful-jobs-history-limit-27897518-4trjv started at 2023-01-16 06:38:00 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.114: INFO: Container c ready: false, restart count 0 Jan 16 06:38:08.114: INFO: all-succeed-b2zfw started at 2023-01-16 06:33:56 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.114: INFO: Container c ready: false, restart count 0 Jan 16 06:38:08.114: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:34:54 +0000 UTC (0+7 container statuses recorded) Jan 16 06:38:08.114: INFO: Container csi-attacher ready: false, restart count 0 Jan 16 06:38:08.114: INFO: Container csi-provisioner ready: false, restart count 0 Jan 16 06:38:08.114: INFO: Container csi-resizer ready: false, restart count 0 Jan 16 06:38:08.114: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 16 06:38:08.114: INFO: Container hostpath ready: false, restart count 0 Jan 16 06:38:08.114: INFO: Container liveness-probe ready: false, restart count 0 Jan 16 06:38:08.114: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 16 06:38:08.114: INFO: netserver-2 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.114: INFO: Container webserver ready: true, restart count 0 Jan 16 06:38:08.114: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-w4v5z started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.114: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.114: INFO: cilium-9tdp4 started at 2023-01-16 06:26:07 +0000 UTC (1+1 container statuses recorded) Jan 16 06:38:08.114: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:38:08.114: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:38:08.114: INFO: rs-vxcbj started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.114: INFO: Container donothing ready: false, restart count 0 Jan 16 06:38:08.114: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7xc54 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.114: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.114: INFO: csi-mockplugin-0 started at 2023-01-16 06:33:16 +0000 UTC (0+3 container statuses recorded) Jan 16 06:38:08.114: INFO: Container csi-provisioner ready: true, restart count 0 Jan 16 06:38:08.114: INFO: Container driver-registrar ready: true, restart count 0 Jan 16 06:38:08.114: INFO: Container mock ready: true, restart count 0 Jan 16 06:38:08.114: INFO: inline-volume-tester-l8h2w started at 2023-01-16 06:34:11 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.114: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 16 06:38:08.114: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5656k started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.114: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.114: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-crnvb started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.114: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.114: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.114: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.114: INFO: ebs-csi-node-8bhxv started at 2023-01-16 06:26:07 +0000 UTC (0+3 container statuses recorded) Jan 16 06:38:08.114: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:38:08.114: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:38:08.114: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:38:08.114: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-lbzbs started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.114: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.114: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-hlz4f started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.114: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.114: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2k7jz started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.114: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.376: INFO: Latency metrics for node i-0ac9adcf8852f0594 Jan 16 06:38:08.376: INFO: Logging node info for node i-0b3e7c1b3ffabe4e1 Jan 16 06:38:08.439: INFO: Node Info: &Node{ObjectMeta:{i-0b3e7c1b3ffabe4e1 f1f7f96e-8f6d-4178-9a45-c16cedbd8510 13607 0 2023-01-16 06:25:59 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b3e7c1b3ffabe4e1 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-0b3e7c1b3ffabe4e1 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.59.231 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b3e7c1b3ffabe4e1"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:33:15 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:33:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0b3e7c1b3ffabe4e1,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:33:17 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:33:17 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:33:17 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:33:17 +0000 UTC,LastTransitionTime:2023-01-16 06:26:13 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.231,},NodeAddress{Type:ExternalIP,Address:52.12.10.229,},NodeAddress{Type:InternalDNS,Address:i-0b3e7c1b3ffabe4e1.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b3e7c1b3ffabe4e1.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-12-10-229.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec26e5d5e832a48110ff0427b389577a,SystemUUID:ec26e5d5-e832-a481-10ff-0427b389577a,BootID:21853215-8129-4e64-b14a-211cf0ee9b99,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d49f269edca5f14a],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d49f269edca5f14a,DevicePath:,},},Config:nil,},} Jan 16 06:38:08.439: INFO: Logging kubelet events for node i-0b3e7c1b3ffabe4e1 Jan 16 06:38:08.504: INFO: Logging pods the kubelet thinks is on node i-0b3e7c1b3ffabe4e1 Jan 16 06:38:08.574: INFO: coredns-559769c974-v2nv5 started at 2023-01-16 06:26:26 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container coredns ready: true, restart count 0 Jan 16 06:38:08.574: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-4nrmv started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.574: INFO: inline-volume-tester-25k5d started at 2023-01-16 06:33:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 16 06:38:08.574: INFO: pod-b3e1f0e9-7784-4982-97d1-edf15e2c898c started at 2023-01-16 06:33:28 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container test-container ready: false, restart count 0 Jan 16 06:38:08.574: INFO: netserver-3 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container webserver ready: false, restart count 0 Jan 16 06:38:08.574: INFO: ebs-csi-node-zg8kr started at 2023-01-16 06:26:00 +0000 UTC (0+3 container statuses recorded) Jan 16 06:38:08.574: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:38:08.574: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:38:08.574: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:38:08.574: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gqf9j started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.574: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nnzrq started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.574: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s2299 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.574: INFO: cilium-l9bsn started at 2023-01-16 06:26:00 +0000 UTC (1+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:38:08.574: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:38:08.574: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ccx9k started at 2023-01-16 06:32:14 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.574: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-fw4ct started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.574: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0 Jan 16 06:38:08.574: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5dxgn started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.574: INFO: rs-5bvsz started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container donothing ready: true, restart count 0 Jan 16 06:38:08.574: INFO: rs-x97fb started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container donothing ready: true, restart count 0 Jan 16 06:38:08.574: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7z54m started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.574: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:08.574: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:38:08.885: INFO: Latency metrics for node i-0b3e7c1b3ffabe4e1 Jan 16 06:38:08.885: INFO: Logging node info for node i-0bcdb98d646a60a68 Jan 16 06:38:08.947: INFO: Node Info: &Node{ObjectMeta:{i-0bcdb98d646a60a68 ef8446c9-df03-44e4-bfed-57c522916505 14924 0 2023-01-16 06:24:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0bcdb98d646a60a68 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.49.7 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bcdb98d646a60a68"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-16 06:24:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-16 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:24:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:24:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:24:54 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-01-16 06:36:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0bcdb98d646a60a68,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.49.7,},NodeAddress{Type:ExternalIP,Address:35.164.29.62,},NodeAddress{Type:InternalDNS,Address:i-0bcdb98d646a60a68.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0bcdb98d646a60a68.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-164-29-62.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2e02b7de327b42fbf146d4a7087a79,SystemUUID:ec2e02b7-de32-7b42-fbf1-46d4a7087a79,BootID:8ddb4053-2513-4e18-b49d-f322ee6d1f6e,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43640493,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42802003,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967313,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 16 06:38:08.948: INFO: Logging kubelet events for node i-0bcdb98d646a60a68 Jan 16 06:38:09.012: INFO: Logging pods the kubelet thinks is on node i-0bcdb98d646a60a68 Jan 16 06:38:09.080: INFO: etcd-manager-events-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:09.080: INFO: Container etcd-manager ready: true, restart count 0 Jan 16 06:38:09.080: INFO: kube-controller-manager-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:09.080: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 16 06:38:09.080: INFO: kube-scheduler-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:09.080: INFO: Container kube-scheduler ready: true, restart count 0 Jan 16 06:38:09.080: INFO: ebs-csi-node-9sg8r started at 2023-01-16 06:24:42 +0000 UTC (0+3 container statuses recorded) Jan 16 06:38:09.080: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:38:09.080: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:38:09.080: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:38:09.080: INFO: dns-controller-69987775c6-8t6m5 started at 2023-01-16 06:24:43 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:09.080: INFO: Container dns-controller ready: true, restart count 0 Jan 16 06:38:09.080: INFO: ebs-csi-controller-5bd98b456f-54tj7 started at 2023-01-16 06:24:43 +0000 UTC (0+5 container statuses recorded) Jan 16 06:38:09.080: INFO: Container csi-attacher ready: true, restart count 0 Jan 16 06:38:09.080: INFO: Container csi-provisioner ready: true, restart count 0 Jan 16 06:38:09.080: INFO: Container csi-resizer ready: true, restart count 0 Jan 16 06:38:09.080: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:38:09.080: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:38:09.080: INFO: etcd-manager-main-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:09.080: INFO: Container etcd-manager ready: true, restart count 0 Jan 16 06:38:09.080: INFO: kube-apiserver-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+2 container statuses recorded) Jan 16 06:38:09.080: INFO: Container healthcheck ready: true, restart count 0 Jan 16 06:38:09.080: INFO: Container kube-apiserver ready: true, restart count 1 Jan 16 06:38:09.080: INFO: cilium-qwpgj started at 2023-01-16 06:24:42 +0000 UTC (1+1 container statuses recorded) Jan 16 06:38:09.080: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:38:09.080: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:38:09.080: INFO: kops-controller-bjgn2 started at 2023-01-16 06:24:42 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:09.080: INFO: Container kops-controller ready: true, restart count 0 Jan 16 06:38:09.080: INFO: aws-cloud-controller-manager-fb8z2 started at 2023-01-16 06:24:42 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:09.080: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 16 06:38:09.080: INFO: cilium-operator-5dd44dc49f-sftdz started at 2023-01-16 06:24:43 +0000 UTC (0+1 container statuses recorded) Jan 16 06:38:09.080: INFO: Container cilium-operator ready: true, restart count 0 Jan 16 06:38:09.329: INFO: Latency metrics for node i-0bcdb98d646a60a68 [DeferCleanup (Each)] [sig-network] Conntrack tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "conntrack-2130" for this suite. �[38;5;243m01/16/23 06:38:09.329�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sNetworking\sGranular\sChecks\:\sServices\sshould\supdate\sendpoints\:\shttp$'
test/e2e/framework/network/utils.go:866
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc0002ee620, {0x75c6f5c, 0x9}, 0xc002c2f590)
test/e2e/framework/network/utils.go:866 +0x1d0
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc0002ee620, 0x7f46bc21b710?)
test/e2e/framework/network/utils.go:763 +0x55
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc0002ee620, 0x3e?)
test/e2e/framework/network/utils.go:778 +0x3e
k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc000ea8690, {0x0, 0x0, 0x7f46e5a2d108?})
test/e2e/framework/network/utils.go:131 +0x125
k8s.io/kubernetes/test/e2e/network.glob..func22.6.11()
test/e2e/network/networking.go:329 +0x36
[BeforeEach] [sig-network] Networking
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/16/23 06:35:01.883�[0m
Jan 16 06:35:01.883: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename nettest �[38;5;243m01/16/23 06:35:01.884�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/16/23 06:35:02.082�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/16/23 06:35:02.212�[0m
[BeforeEach] [sig-network] Networking
test/e2e/framework/metrics/init/init.go:31
[It] should update endpoints: http
test/e2e/network/networking.go:328
�[1mSTEP:�[0m Performing setup for networking test in namespace nettest-5446 �[38;5;243m01/16/23 06:35:02.341�[0m
�[1mSTEP:�[0m creating a selector �[38;5;243m01/16/23 06:35:02.341�[0m
�[1mSTEP:�[0m Creating the service pods in kubernetes �[38;5;243m01/16/23 06:35:02.342�[0m
Jan 16 06:35:02.342: INFO: Waiting up to 10m0s for all (but 0) nodes to be schedulable
Jan 16 06:35:02.752: INFO: Waiting up to 5m0s for pod "netserver-0" in namespace "nettest-5446" to be "running and ready"
Jan 16 06:35:02.818: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 65.124125ms
Jan 16 06:35:02.818: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:04.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2.130666528s
Jan 16 06:35:04.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:06.885: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4.132532484s
Jan 16 06:35:06.885: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:08.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 6.131572807s
Jan 16 06:35:08.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:10.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 8.130742999s
Jan 16 06:35:10.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:12.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 10.131315092s
Jan 16 06:35:12.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:14.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 12.131565417s
Jan 16 06:35:14.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:16.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 14.130934492s
Jan 16 06:35:16.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:18.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 16.130708626s
Jan 16 06:35:18.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:20.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 18.131193783s
Jan 16 06:35:20.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:22.887: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 20.135016631s
Jan 16 06:35:22.887: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:24.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 22.130648525s
Jan 16 06:35:24.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:26.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 24.131315694s
Jan 16 06:35:26.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:28.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 26.130612985s
Jan 16 06:35:28.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:30.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 28.131148427s
Jan 16 06:35:30.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:32.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 30.130804183s
Jan 16 06:35:32.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:34.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 32.130549649s
Jan 16 06:35:34.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:36.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 34.13141775s
Jan 16 06:35:36.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:38.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 36.130507966s
Jan 16 06:35:38.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:40.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 38.130487291s
Jan 16 06:35:40.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:42.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 40.131475717s
Jan 16 06:35:42.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:44.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 42.131717437s
Jan 16 06:35:44.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:46.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 44.130493685s
Jan 16 06:35:46.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:48.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 46.13055992s
Jan 16 06:35:48.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:50.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 48.131530481s
Jan 16 06:35:50.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:52.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 50.13164744s
Jan 16 06:35:52.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:54.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 52.131504411s
Jan 16 06:35:54.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:56.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 54.131214187s
Jan 16 06:35:56.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:35:58.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 56.131061009s
Jan 16 06:35:58.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:00.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 58.130821132s
Jan 16 06:36:00.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:02.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.131626673s
Jan 16 06:36:02.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:04.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.130843067s
Jan 16 06:36:04.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:06.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.131574569s
Jan 16 06:36:06.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:08.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.131317935s
Jan 16 06:36:08.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:10.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.131675205s
Jan 16 06:36:10.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:12.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.131230439s
Jan 16 06:36:12.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:14.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.130712579s
Jan 16 06:36:14.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:16.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.131048927s
Jan 16 06:36:16.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:18.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.130928523s
Jan 16 06:36:18.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:20.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.130992659s
Jan 16 06:36:20.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:22.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.131085256s
Jan 16 06:36:22.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:24.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.130469052s
Jan 16 06:36:24.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:26.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.131166934s
Jan 16 06:36:26.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:28.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.131014804s
Jan 16 06:36:28.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:30.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.131342876s
Jan 16 06:36:30.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:32.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.130518149s
Jan 16 06:36:32.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:34.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.130996367s
Jan 16 06:36:34.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:36.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.131191675s
Jan 16 06:36:36.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:38.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.130633354s
Jan 16 06:36:38.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:40.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.131196492s
Jan 16 06:36:40.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:42.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.130633847s
Jan 16 06:36:42.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:44.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.131187395s
Jan 16 06:36:44.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:46.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.131434663s
Jan 16 06:36:46.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:48.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.130866922s
Jan 16 06:36:48.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:50.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.131326615s
Jan 16 06:36:50.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:52.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.131358772s
Jan 16 06:36:52.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:54.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.130573105s
Jan 16 06:36:54.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:56.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.131488741s
Jan 16 06:36:56.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:36:58.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.130954052s
Jan 16 06:36:58.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:00.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.131140643s
Jan 16 06:37:00.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:02.891: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.138388644s
Jan 16 06:37:02.891: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:04.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.130741737s
Jan 16 06:37:04.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:06.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.131728622s
Jan 16 06:37:06.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:08.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.131290905s
Jan 16 06:37:08.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:10.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.131311673s
Jan 16 06:37:10.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:12.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.13063123s
Jan 16 06:37:12.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:14.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.130503475s
Jan 16 06:37:14.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:16.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.131262218s
Jan 16 06:37:16.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:18.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.130719326s
Jan 16 06:37:18.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:20.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.131613951s
Jan 16 06:37:20.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:22.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.131444418s
Jan 16 06:37:22.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:24.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.130736966s
Jan 16 06:37:24.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:26.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.131451874s
Jan 16 06:37:26.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:28.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.130553275s
Jan 16 06:37:28.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:30.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.130489712s
Jan 16 06:37:30.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:32.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.131371382s
Jan 16 06:37:32.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:34.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.130439251s
Jan 16 06:37:34.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:36.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.131315709s
Jan 16 06:37:36.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:38.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.130652817s
Jan 16 06:37:38.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:40.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.131548095s
Jan 16 06:37:40.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:42.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.13064976s
Jan 16 06:37:42.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:44.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.130806442s
Jan 16 06:37:44.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:46.885: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.132409262s
Jan 16 06:37:46.885: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:48.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.13055632s
Jan 16 06:37:48.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:50.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.130603835s
Jan 16 06:37:50.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:52.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.130346709s
Jan 16 06:37:52.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:54.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.131346817s
Jan 16 06:37:54.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:56.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.130666816s
Jan 16 06:37:56.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:37:58.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.131131734s
Jan 16 06:37:58.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:00.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.131284929s
Jan 16 06:38:00.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:02.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.130477249s
Jan 16 06:38:02.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:04.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.131173679s
Jan 16 06:38:04.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:06.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.130406087s
Jan 16 06:38:06.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:08.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.131338662s
Jan 16 06:38:08.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:10.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.131000399s
Jan 16 06:38:10.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:12.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.131415169s
Jan 16 06:38:12.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:14.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.131252731s
Jan 16 06:38:14.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:16.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.131634849s
Jan 16 06:38:16.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:18.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.131758658s
Jan 16 06:38:18.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:20.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.130696887s
Jan 16 06:38:20.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:22.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.130980318s
Jan 16 06:38:22.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:24.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.130611083s
Jan 16 06:38:24.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:26.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.131364856s
Jan 16 06:38:26.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:28.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.130500013s
Jan 16 06:38:28.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:30.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.131188015s
Jan 16 06:38:30.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:32.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.131231377s
Jan 16 06:38:32.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:34.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.130499948s
Jan 16 06:38:34.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:36.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.13111451s
Jan 16 06:38:36.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:38.885: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.132224427s
Jan 16 06:38:38.885: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:40.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.131338847s
Jan 16 06:38:40.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:42.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.131299372s
Jan 16 06:38:42.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:44.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.131405332s
Jan 16 06:38:44.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:46.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.13064904s
Jan 16 06:38:46.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:48.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.130743876s
Jan 16 06:38:48.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:50.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.13073388s
Jan 16 06:38:50.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:52.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.131473523s
Jan 16 06:38:52.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:54.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.130635207s
Jan 16 06:38:54.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:56.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.131564541s
Jan 16 06:38:56.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:38:58.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.131438212s
Jan 16 06:38:58.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:00.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.130518238s
Jan 16 06:39:00.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:02.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.131272832s
Jan 16 06:39:02.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:04.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.130626234s
Jan 16 06:39:04.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:06.979: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.226462163s
Jan 16 06:39:06.979: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:08.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.130627383s
Jan 16 06:39:08.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:10.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.131234782s
Jan 16 06:39:10.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:12.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.131365498s
Jan 16 06:39:12.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:14.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.131301492s
Jan 16 06:39:14.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:16.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.130930628s
Jan 16 06:39:16.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:18.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.131403878s
Jan 16 06:39:18.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:20.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.130510284s
Jan 16 06:39:20.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:22.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.130642182s
Jan 16 06:39:22.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:24.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.130720838s
Jan 16 06:39:24.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:26.883: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.13055651s
Jan 16 06:39:26.883: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:28.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.131292258s
Jan 16 06:39:28.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:30.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.131388221s
Jan 16 06:39:30.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:32.887: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.134063226s
Jan 16 06:39:32.887: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:34.885: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.132538816s
Jan 16 06:39:34.885: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:36.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.131569774s
Jan 16 06:39:36.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:38.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.13135613s
Jan 16 06:39:38.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:40.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.131657404s
Jan 16 06:39:40.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:42.884: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.131435079s
Jan 16 06:39:42.884: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 16 06:39:44.884: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 4m42.131203598s
Jan 16 06:39:44.884: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 16 06:39:46.886: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 4m44.133517664s
Jan 16 06:39:46.886: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 16 06:39:48.891: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 4m46.138643733s
Jan 16 06:39:48.891: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 16 06:39:50.885: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 4m48.132778758s
Jan 16 06:39:50.885: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 16 06:39:52.884: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 4m50.131492087s
Jan 16 06:39:52.884: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 16 06:39:54.883: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 4m52.13103112s
Jan 16 06:39:54.884: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 16 06:39:56.908: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 4m54.155731531s
Jan 16 06:39:56.908: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 16 06:39:58.884: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 4m56.131354737s
Jan 16 06:39:58.884: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 16 06:40:00.885: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 4m58.132993229s
Jan 16 06:40:00.885: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 16 06:40:02.888: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 5m0.13570132s
Jan 16 06:40:02.888: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 16 06:40:02.954: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 5m0.201765012s
Jan 16 06:40:02.954: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 16 06:40:02.955: INFO: Unexpected error:
<*pod.timeoutError | 0xc002c2ff20>: {
msg: "timed out while waiting for pod nettest-5446/netserver-0 to be running and ready",
observedObjects: [
<*v1.Pod | 0xc00116ad80>{
TypeMeta: {Kind: "", APIVersion: ""},
ObjectMeta: {
Name: "netserver-0",
GenerateName: "",
Namespace: "nettest-5446",
SelfLink: "",
UID: "13d3eb69-50e4-4654-b4e4-eec1f38c9db7",
ResourceVersion: "16556",
Generation: 0,
CreationTimestamp: {
Time: {
wall: 0,
ext: 63809447702,
loc: {
name: "Local",
zone: [
{name: "UTC", offset: 0, isDST: false},
],
tx: [
{
when: -576460752303423488,
index: 0,
isstd: false,
isutc: false,
},
],
extend: "UTC0",
cacheStart: 9223372036854775807,
cacheEnd: 9223372036854775807,
cacheZone: {name: "UTC", offset: 0, isDST: false},
},
},
},
DeletionTimestamp: nil,
DeletionGracePeriodSeconds: nil,
Labels: {
"selector-369842b2-2099-417f-88ce-4ba8ae1530f5": "true",
},
Annotations: nil,
OwnerReferences: nil,
Finalizers: nil,
ManagedFields: [
{
Manager: "e2e.test",
Operation: "Update",
APIVersion: "v1",
Time: {
Time: {
wall: 0,
ext: 63809447702,
loc: {
name: "Local",
zone: [...],
tx: [...],
extend: "UTC0",
cacheStart: 9223372036854775807,
cacheEnd: 9223372036854775807,
cacheZone: {name: ..., offset: ..., isDST: ...},
},
},
},
FieldsType: "FieldsV1",
FieldsV1: {
Raw: "{\"f:metadata\":{\"f:labels\":{\".\":{},\"f:selector-369842b2-2099-417f-88ce-4ba8ae1530f5\":{}}},\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"webserver\\\"}\":{\".\":{},\"f:args\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:livenessProbe\":{\".\":{},\"f:failureThreshold\":{},\"f:httpGet\":{\".\":{},\"f:path\":{},\"f:port\":{},\"f:scheme\":{}},\"f:initialDelaySeconds\":{},\"f:periodSeconds\":{},\"f:successThreshold\":{},\"f:timeoutSeconds\":{}},\"f:name\":{},\"f:ports\":{\".\":{},\"k:{\\\"containerPort\\\":8081,\\\"protocol\\\":\\\"UDP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:name\":{},\"f:protocol\":{}},\"k:{\\\"containerPort\\\":8083,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:name\":{},\"f:protocol\":{}}},\"f:readinessProbe\":{\".\":{},\"f:failureThreshold\":{},\"f:httpGet\":{\".\":{},\"f:path\":{},\"f:port\":{},\"f:scheme\":{}},\"f:initialDelaySeconds\":{},\"f:periodSeconds\":{},\"f:successThreshold\":{},\"f:timeoutSeconds\":{}},\"f:resources\":{},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:nodeSelector\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:terminationGracePeriodSeconds\":{}}}",
},
Subresource: "",
},
{
Manager: "kubelet",
Operation: "Update",
APIVersion: "v1",
Time: {
Time: {
wall: 0,
ext: 63809447983,
loc: {
name: "Local",
zone: [...],
tx: [...],
extend: "UTC0",
cacheStart: 9223372036854775807,
cacheEnd: 9223372036854775807,
cacheZone: {name: ..., offset: ..., isDST: ...},
},
},
},
FieldsType: "FieldsV1",
FieldsV1: {
Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:phase\":{},\"f:podIP\":{},\"f:podIPs\":{\".\":{},\"k:{\\\"ip\\\":\\\"172.20.56.109\\\"}\":{\".\":{},\"f:ip\":{}}},\"f:startTime\":{}}}",
},
Subresource: "status",
},
],
},
Spec: {
Volumes: [
{
Name: "kube-api-access-8xp55",
VolumeSource: {
HostPath: nil,
EmptyDir: nil,
GCEPersistentDisk: nil,
AWSElasticBlockStore: nil,
GitRepo: nil,
Secret: nil,
NFS: nil,
ISCSI: nil,
Glusterfs: nil,
PersistentVolumeClaim: nil,
RBD: nil,
FlexVolume: nil,
Cinder: nil,
CephFS: nil,
Flocker: nil,
DownwardAPI: nil,
FC: nil,
AzureFile: nil,
ConfigMap: nil,
VsphereVolume: nil,
Quobyte: nil,
AzureDisk: nil,
PhotonPersistentDisk: nil,
Projected: {
Sources: [
{
Secret: ...,
DownwardAPI: ...,
ConfigMap: ...,
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
Jan 16 06:40:02.955: FAIL: timed out while waiting for pod nettest-5446/netserver-0 to be running and ready
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc0002ee620, {0x75c6f5c, 0x9}, 0xc002c2f590)
test/e2e/framework/network/utils.go:866 +0x1d0
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc0002ee620, 0x7f46bc21b710?)
test/e2e/framework/network/utils.go:763 +0x55
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc0002ee620, 0x3e?)
test/e2e/framework/network/utils.go:778 +0x3e
k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc000ea8690, {0x0, 0x0, 0x7f46e5a2d108?})
test/e2e/framework/network/utils.go:131 +0x125
k8s.io/kubernetes/test/e2e/network.glob..func22.6.11()
test/e2e/network/networking.go:329 +0x36
[AfterEach] [sig-network] Networking
test/e2e/framework/node/init/init.go:32
Jan 16 06:40:02.956: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-network] Networking
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-network] Networking
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/16/23 06:40:03.022�[0m
�[1mSTEP:�[0m Collecting events from namespace "nettest-5446". �[38;5;243m01/16/23 06:40:03.022�[0m
�[1mSTEP:�[0m Found 55 events. �[38;5;243m01/16/23 06:40:03.092�[0m
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:02 +0000 UTC - event for netserver-0: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "00244a387ff93be83214bf9a41b740bb9c0507541f7a8ffbbdae6067ed480479": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:02 +0000 UTC - event for netserver-0: {default-scheduler } Scheduled: Successfully assigned nettest-5446/netserver-0 to i-09cbbd5e7917c52c5
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:02 +0000 UTC - event for netserver-1: {default-scheduler } Scheduled: Successfully assigned nettest-5446/netserver-1 to i-09cdc5f20702b22f2
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:02 +0000 UTC - event for netserver-2: {default-scheduler } Scheduled: Successfully assigned nettest-5446/netserver-2 to i-0ac9adcf8852f0594
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:02 +0000 UTC - event for netserver-3: {default-scheduler } Scheduled: Successfully assigned nettest-5446/netserver-3 to i-0b3e7c1b3ffabe4e1
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:03 +0000 UTC - event for netserver-1: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4e1f10be6367cfb8c22d13474baba8fee01bb3880bae358776022bed4be43ffe": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:03 +0000 UTC - event for netserver-2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3f42cba8a86c6bf337270b42d4169c7824cf712ca080c4f768942e2fa4e6e68e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:03 +0000 UTC - event for netserver-3: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "fe4946c3661e61a864d1223fcc983d5946b259adf2b1417f282dba665ace99d8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:14 +0000 UTC - event for netserver-0: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6ce2624c54304cb3dab20d80ce78cc7e1c6867625a63d016ab946f8449d7c990": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:14 +0000 UTC - event for netserver-3: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b2824e4ae3344a274372a9cd335ace02029f05528c82e05c2c7d6429e2cc3d7f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:16 +0000 UTC - event for netserver-2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "70f1571169dc2827f783a83cfb10b6c0061930193f4d7990190db2af51145674": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:17 +0000 UTC - event for netserver-1: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d893db58598bd14a0e9b466fff02511dd6610de50ba0eca60e729268404fecbc": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:26 +0000 UTC - event for netserver-3: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "73142c335321db23ea23e14efba1a4209afd98357742f27513fa6d85a4851e8f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:27 +0000 UTC - event for netserver-0: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "257b05b44edc346801e0125ee9c4a763175952bafc1b3bc6c483eadf82845fa6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:28 +0000 UTC - event for netserver-2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "44202001514de3ee49c5fc1fc0c1b36517e30a7f552b28af9bd862695fd379c3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:32 +0000 UTC - event for netserver-1: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b62113cd00df0aabfb827b34d0597d583a83740e3bf5dc8a8d836fca96a34428": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:40 +0000 UTC - event for netserver-0: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "866166861d3808dd5dfe23cf44f71b28673fa20aa22329803978ef777765c202": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:40 +0000 UTC - event for netserver-2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "18e9e3259440b6650776f1f3baedcb119e7413678139c41093d3825a631162f6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:40 +0000 UTC - event for netserver-3: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e168b81a8a003c8d6c2d1d4b1fde1b4316766795aa0f8b97dbdad4f73323c3a1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:45 +0000 UTC - event for netserver-1: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3fc9b8dc75fbb19e494679bece12e894e8fb4bfc0015de8a323ceb9a1c686639": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:51 +0000 UTC - event for netserver-2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3e025c5056d0ec3b3cef22e1c91b7d737e1d57b84e971e6522964f07cda2c836": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:54 +0000 UTC - event for netserver-3: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5b0ece363e95390b133b0aa761e3b62209f21ec0be4ff8ffa906393024b36043": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:35:55 +0000 UTC - event for netserver-0: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "13ac39466f5e895b76020af172dff849990c7c28336b9fb05283bf48f6cdd476": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:00 +0000 UTC - event for netserver-1: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a166e0acbf8fa9d81ffd6854377825f01131f9ceb3292144393c45f47c48578c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:02 +0000 UTC - event for netserver-2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "fab1958e3e9b08b4fb76afedc99ed572243cccb57aa2c54d9a4c194f2e9f4ecf": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:06 +0000 UTC - event for netserver-0: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "dae377ed3b984ecd7f76e88d5e3392c68d56a121b5765f8c56ab9cdd7a944769": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:08 +0000 UTC - event for netserver-3: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "db4c5a4d864894c1dddb3f7a038550a2fd0ccc23064dac6a8d2770da145a5c15": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:14 +0000 UTC - event for netserver-1: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d24bdbaeec542d29aab4c765db53a028d12350666cb7852007a75f6dcd06d2bc": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:15 +0000 UTC - event for netserver-2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "81c92ca3203e1b8f8b339b07a7ed2465a8487943266ecf83e8d3d5c1251c00a1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:20 +0000 UTC - event for netserver-3: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "20c559e42c8b8cb933dd19e5f4b029cbea38f1ef57c5170514da15df8219a16b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:21 +0000 UTC - event for netserver-0: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "79d4ba61b02f130a28f8146ee32577e8ac35ad1c4099c0e766a67618eeb71d7c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:29 +0000 UTC - event for netserver-1: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "76b2ee60d2fcc1f2c6e3d0df2b8496041a5ab93ca121b6267b6e3802b06d6862": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:29 +0000 UTC - event for netserver-2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6378f71a817ae63454903edff355b3b6cf11d89d0f590c844776a5bc6bd2184b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:32 +0000 UTC - event for netserver-3: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ebcf42fe3d58fb5940612eb84205972e2b596100c742cc10ba08c1abf24f24e0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:35 +0000 UTC - event for netserver-0: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1a337b94c7e9e003ac1483f00f6a6a9fef00ddf1048b7b74215a24dea8538765": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:41 +0000 UTC - event for netserver-1: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c17643a095f7e08a6dd7366fa78133dc9b607955eac9bde6156b314d5b34b820": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:42 +0000 UTC - event for netserver-2: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6199284fe81230d4719d5eb16a96255c917a2070d82ca0f35a1fb053159db425": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:47 +0000 UTC - event for netserver-3: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ae55cfafb66603563cf92b2c52d5953a5d08931330b26c5b62f0a0f6184e6016": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:48 +0000 UTC - event for netserver-0: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "627a8033b5f5f4d29c152b6b5cc4d55f59349448385b4ad29f9def21050c1c90": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:55 +0000 UTC - event for netserver-1: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "88841fd6471670dcc4abda04de10888aeec8bb24fbeb7d59df8f8d242bee6a8c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:56 +0000 UTC - event for netserver-2: {kubelet i-0ac9adcf8852f0594} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:56 +0000 UTC - event for netserver-2: {kubelet i-0ac9adcf8852f0594} Created: Created container webserver
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:56 +0000 UTC - event for netserver-2: {kubelet i-0ac9adcf8852f0594} Started: Started container webserver
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:36:59 +0000 UTC - event for netserver-3: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "882c3138eea3b62965dff01088bf9ee2f9315b7bd4b616bcaf1c46ff391f0230": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:37:02 +0000 UTC - event for netserver-0: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6018801f2e87d052e7bb4e6f580e7cf97bd9c1895a663bab631a798e3a0734a1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:37:08 +0000 UTC - event for netserver-1: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4556daaa765f3e26843772d9638d959648b2a5f0c55d2e3a468a17076afa04c5": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:39:38 +0000 UTC - event for netserver-0: {kubelet i-09cbbd5e7917c52c5} Created: Created container webserver
Jan 16 06:40:03.092: INFO: At 2023-01-16 06:39:38 +0000 UTC - event for netserver-0: {kubelet i-09cbbd5e7917c52c5} Started: Started container webserver
Jan 16 06:40:03.093: INFO: At 2023-01-16 06:39:38 +0000 UTC - event for netserver-0: {kubelet i-09cbbd5e7917c52c5} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Jan 16 06:40:03.093: INFO: At 2023-01-16 06:39:42 +0000 UTC - event for netserver-1: {kubelet i-09cdc5f20702b22f2} Started: Started container webserver
Jan 16 06:40:03.093: INFO: At 2023-01-16 06:39:42 +0000 UTC - event for netserver-1: {kubelet i-09cdc5f20702b22f2} Created: Created container webserver
Jan 16 06:40:03.093: INFO: At 2023-01-16 06:39:42 +0000 UTC - event for netserver-1: {kubelet i-09cdc5f20702b22f2} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Jan 16 06:40:03.093: INFO: At 2023-01-16 06:39:42 +0000 UTC - event for netserver-3: {kubelet i-0b3e7c1b3ffabe4e1} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Jan 16 06:40:03.093: INFO: At 2023-01-16 06:39:42 +0000 UTC - event for netserver-3: {kubelet i-0b3e7c1b3ffabe4e1} Created: Created container webserver
Jan 16 06:40:03.093: INFO: At 2023-01-16 06:39:42 +0000 UTC - event for netserver-3: {kubelet i-0b3e7c1b3ffabe4e1} Started: Started container webserver
Jan 16 06:40:03.159: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 16 06:40:03.159: INFO: netserver-0 i-09cbbd5e7917c52c5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:02 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:02 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:02 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:02 +0000 UTC }]
Jan 16 06:40:03.159: INFO: netserver-1 i-09cdc5f20702b22f2 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:02 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:02 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:02 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:02 +0000 UTC }]
Jan 16 06:40:03.159: INFO: netserver-2 i-0ac9adcf8852f0594 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:02 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:37:13 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:37:13 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:02 +0000 UTC }]
Jan 16 06:40:03.159: INFO: netserver-3 i-0b3e7c1b3ffabe4e1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:02 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:02 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:02 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:35:02 +0000 UTC }]
Jan 16 06:40:03.159: INFO:
Jan 16 06:40:03.516: INFO:
Logging node info for node i-09cbbd5e7917c52c5
Jan 16 06:40:03.581: INFO: Node Info: &Node{ObjectMeta:{i-09cbbd5e7917c52c5 516d50c9-4cfe-4b03-9036-fb7eaefbf36d 17459 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cbbd5e7917c52c5 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cbbd5e7917c52c5 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.43.77 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-3020":"i-09cbbd5e7917c52c5","csi-hostpath-volume-expand-7657":"i-09cbbd5e7917c52c5","csi-mock-csi-mock-volumes-8264":"i-09cbbd5e7917c52c5","ebs.csi.aws.com":"i-09cbbd5e7917c52c5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:39:54 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:39:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cbbd5e7917c52c5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:39:58 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:39:58 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:39:58 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:39:58 +0000 UTC,LastTransitionTime:2023-01-16 06:26:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.77,},NodeAddress{Type:ExternalIP,Address:52.25.36.68,},NodeAddress{Type:InternalDNS,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-36-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2589b28acc26043d6f13727e02d530,SystemUUID:ec2589b2-8acc-2604-3d6f-13727e02d530,BootID:f8305071-d74c-44c0-a267-8cc84ea9ad7b,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-3020^9489448c-9567-11ed-b056-f2aad9c35bb6 kubernetes.io/csi/csi-hostpath-volume-expand-7657^8de0dd9f-9568-11ed-a4cd-c26c2f99fd12 kubernetes.io/csi/csi-mock-csi-mock-volumes-8264^94d04e32-9568-11ed-8cd2-f2d13f722f77],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-3020^9489448c-9567-11ed-b056-f2aad9c35bb6,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-7657^8de0dd9f-9568-11ed-a4cd-c26c2f99fd12,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-8264^94d04e32-9568-11ed-8cd2-f2d13f722f77,DevicePath:,},},Config:nil,},}
Jan 16 06:40:03.582: INFO:
Logging kubelet events for node i-09cbbd5e7917c52c5
Jan 16 06:40:03.649: INFO:
Logging pods the kubelet thinks is on node i-09cbbd5e7917c52c5
Jan 16 06:40:03.724: INFO: netserver-0 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:03.724: INFO: Container webserver ready: false, restart count 0
Jan 16 06:40:03.724: INFO: ebs-csi-node-smxsx started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:40:03.724: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:40:03.724: INFO: coredns-autoscaler-7cb5c5b969-lt7sb started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:03.724: INFO: Container autoscaler ready: true, restart count 0
Jan 16 06:40:03.724: INFO: rs-qcmvk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:03.724: INFO: Container donothing ready: true, restart count 0
Jan 16 06:40:03.724: INFO: rs-9ttnx started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:03.724: INFO: Container donothing ready: true, restart count 0
Jan 16 06:40:03.724: INFO: hostexec-i-09cbbd5e7917c52c5-nnx6b started at 2023-01-16 06:39:50 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:03.724: INFO: Container agnhost-container ready: true, restart count 0
Jan 16 06:40:03.724: INFO: inline-volume-tester-5b6t4 started at 2023-01-16 06:32:43 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:03.724: INFO: Container csi-volume-tester ready: true, restart count 0
Jan 16 06:40:03.724: INFO: csi-mockplugin-0 started at 2023-01-16 06:39:30 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:40:03.724: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container driver-registrar ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container mock ready: true, restart count 0
Jan 16 06:40:03.724: INFO: csi-mockplugin-attacher-0 started at 2023-01-16 06:39:30 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:03.724: INFO: Container csi-attacher ready: true, restart count 0
Jan 16 06:40:03.724: INFO: coredns-559769c974-bxl2p started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:03.724: INFO: Container coredns ready: true, restart count 0
Jan 16 06:40:03.724: INFO: pod-0d8fc412-31df-4944-9b57-ba4d4b81d814 started at 2023-01-16 06:39:42 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:03.724: INFO: Container write-pod ready: false, restart count 0
Jan 16 06:40:03.724: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:36:13 +0000 UTC (0+7 container statuses recorded)
Jan 16 06:40:03.724: INFO: Container csi-attacher ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container csi-resizer ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container hostpath ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:40:03.724: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:32:40 +0000 UTC (0+7 container statuses recorded)
Jan 16 06:40:03.724: INFO: Container csi-attacher ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container csi-resizer ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container hostpath ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:40:03.724: INFO: hostexec-i-09cbbd5e7917c52c5-4727c started at <nil> (0+0 container statuses recorded)
Jan 16 06:40:03.724: INFO: pvc-volume-tester-p8gm5 started at <nil> (0+0 container statuses recorded)
Jan 16 06:40:03.724: INFO: cilium-xxhgc started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:40:03.724: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:40:03.724: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:40:03.724: INFO: hostexec-i-09cbbd5e7917c52c5-tg2d8 started at <nil> (0+0 container statuses recorded)
Jan 16 06:40:04.071: INFO:
Latency metrics for node i-09cbbd5e7917c52c5
Jan 16 06:40:04.071: INFO:
Logging node info for node i-09cdc5f20702b22f2
Jan 16 06:40:04.137: INFO: Node Info: &Node{ObjectMeta:{i-09cdc5f20702b22f2 684a930e-dce6-418d-b0e9-bf389ce1a6d1 17414 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cdc5f20702b22f2 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cdc5f20702b22f2 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.120 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cdc5f20702b22f2"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:39:56 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:39:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cdc5f20702b22f2,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:39:58 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:39:58 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:39:58 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:39:58 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.120,},NodeAddress{Type:ExternalIP,Address:52.25.16.21,},NodeAddress{Type:InternalDNS,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-16-21.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec20ae9de53dd52df737771e3cb612ee,SystemUUID:ec20ae9d-e53d-d52d-f737-771e3cb612ee,BootID:731f2d80-218c-42a6-8438-95aae6d5c60c,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-079387d474e82bcd0],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-079387d474e82bcd0,DevicePath:,},},Config:nil,},}
Jan 16 06:40:04.137: INFO:
Logging kubelet events for node i-09cdc5f20702b22f2
Jan 16 06:40:04.205: INFO:
Logging pods the kubelet thinks is on node i-09cdc5f20702b22f2
Jan 16 06:40:04.342: INFO: proxy-service-pvd9c-4hvrh started at <nil> (0+0 container statuses recorded)
Jan 16 06:40:04.342: INFO: netserver-1 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.342: INFO: Container webserver ready: false, restart count 0
Jan 16 06:40:04.342: INFO: inline-volume-tester2-4pt6h started at 2023-01-16 06:39:53 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.342: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 16 06:40:04.342: INFO: cilium-wp5d7 started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:40:04.342: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:40:04.342: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:40:04.342: INFO: verify-service-up-host-exec-pod started at 2023-01-16 06:39:45 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.342: INFO: Container agnhost-container ready: true, restart count 0
Jan 16 06:40:04.342: INFO: all-pods-removed-6lcv7 started at 2023-01-16 06:33:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.342: INFO: Container c ready: true, restart count 0
Jan 16 06:40:04.342: INFO: externalsvc-fs44s started at <nil> (0+0 container statuses recorded)
Jan 16 06:40:04.342: INFO: service-deletion-9hpkv started at 2023-01-16 06:35:33 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.342: INFO: Container service-deletion ready: true, restart count 0
Jan 16 06:40:04.342: INFO: verify-service-up-exec-pod-gj7pd started at 2023-01-16 06:39:49 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.342: INFO: Container agnhost-container ready: false, restart count 0
Jan 16 06:40:04.342: INFO: ebs-csi-node-gk7tk started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:40:04.342: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:40:04.342: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:40:04.342: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:40:04.342: INFO: rs-qzzlk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.342: INFO: Container donothing ready: true, restart count 0
Jan 16 06:40:04.342: INFO: all-pods-removed-hgfff started at 2023-01-16 06:33:12 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.342: INFO: Container c ready: true, restart count 0
Jan 16 06:40:04.342: INFO: dns-test-164a87cf-8613-4195-9244-1bac7a28d7df started at 2023-01-16 06:39:10 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:40:04.342: INFO: Container jessie-querier ready: true, restart count 0
Jan 16 06:40:04.342: INFO: Container querier ready: true, restart count 0
Jan 16 06:40:04.342: INFO: Container webserver ready: true, restart count 0
Jan 16 06:40:04.636: INFO:
Latency metrics for node i-09cdc5f20702b22f2
Jan 16 06:40:04.636: INFO:
Logging node info for node i-0ac9adcf8852f0594
Jan 16 06:40:04.702: INFO: Node Info: &Node{ObjectMeta:{i-0ac9adcf8852f0594 5c88bf26-14b3-4940-8136-fa11c560695e 17666 0 2023-01-16 06:26:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0ac9adcf8852f0594 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-0ac9adcf8852f0594 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.221 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-8638":"i-0ac9adcf8852f0594","csi-mock-csi-mock-volumes-2633":"i-0ac9adcf8852f0594","csi-mock-csi-mock-volumes-4883":"i-0ac9adcf8852f0594","csi-mock-csi-mock-volumes-7038":"i-0ac9adcf8852f0594","ebs.csi.aws.com":"i-0ac9adcf8852f0594"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:39:40 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:40:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0ac9adcf8852f0594,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:39:43 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:39:43 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:39:43 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:39:43 +0000 UTC,LastTransitionTime:2023-01-16 06:26:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.221,},NodeAddress{Type:ExternalIP,Address:35.90.38.176,},NodeAddress{Type:InternalDNS,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-90-38-176.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2092b4f74388fb9645e6987d21584e,SystemUUID:ec2092b4-f743-88fb-9645-e6987d21584e,BootID:6319a23e-d063-4cef-b18e-f1906415b7a3,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-8638^8c535a7c-9568-11ed-9f81-46f5747c816b kubernetes.io/csi/ebs.csi.aws.com^vol-051cd447c54259a6e kubernetes.io/csi/ebs.csi.aws.com^vol-0c3cdffdafd49e4d9],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-8638^8c535a7c-9568-11ed-9f81-46f5747c816b,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-051cd447c54259a6e,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0c3cdffdafd49e4d9,DevicePath:,},},Config:nil,},}
Jan 16 06:40:04.702: INFO:
Logging kubelet events for node i-0ac9adcf8852f0594
Jan 16 06:40:04.770: INFO:
Logging pods the kubelet thinks is on node i-0ac9adcf8852f0594
Jan 16 06:40:04.843: INFO: netserver-2 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container webserver ready: true, restart count 0
Jan 16 06:40:04.843: INFO: csi-mockplugin-0 started at 2023-01-16 06:33:16 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:40:04.843: INFO: Container driver-registrar ready: true, restart count 0
Jan 16 06:40:04.843: INFO: Container mock ready: true, restart count 0
Jan 16 06:40:04.843: INFO: csi-mockplugin-0 started at 2023-01-16 06:39:45 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container csi-provisioner ready: false, restart count 0
Jan 16 06:40:04.843: INFO: Container driver-registrar ready: false, restart count 0
Jan 16 06:40:04.843: INFO: Container mock ready: false, restart count 0
Jan 16 06:40:04.843: INFO: pvc-volume-tester-48722 started at 2023-01-16 06:35:01 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container volume-tester ready: true, restart count 0
Jan 16 06:40:04.843: INFO: csi-mockplugin-attacher-0 started at 2023-01-16 06:38:36 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container csi-attacher ready: true, restart count 0
Jan 16 06:40:04.843: INFO: csi-mockplugin-0 started at 2023-01-16 06:38:36 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:40:04.843: INFO: Container driver-registrar ready: true, restart count 0
Jan 16 06:40:04.843: INFO: Container mock ready: true, restart count 0
Jan 16 06:40:04.843: INFO: pod-344c7a7e-7263-4a49-ae8a-f5bd982839c6 started at 2023-01-16 06:39:39 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container write-pod ready: false, restart count 0
Jan 16 06:40:04.843: INFO: pod-secrets-759457eb-457a-4544-b365-ae08cf254e54 started at 2023-01-16 06:39:51 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container secret-env-test ready: false, restart count 0
Jan 16 06:40:04.843: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:34:54 +0000 UTC (0+7 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container csi-attacher ready: true, restart count 0
Jan 16 06:40:04.843: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:40:04.843: INFO: Container csi-resizer ready: true, restart count 0
Jan 16 06:40:04.843: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 16 06:40:04.843: INFO: Container hostpath ready: true, restart count 0
Jan 16 06:40:04.843: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:40:04.843: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:40:04.843: INFO: externalsvc-rg6gg started at 2023-01-16 06:39:52 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container externalsvc ready: false, restart count 0
Jan 16 06:40:04.843: INFO: pod-logs-websocket-f7fd16d2-6330-41d5-bd34-4939ca38a8bc started at 2023-01-16 06:38:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container main ready: true, restart count 0
Jan 16 06:40:04.843: INFO: cilium-9tdp4 started at 2023-01-16 06:26:07 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:40:04.843: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:40:04.843: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:40:04.843: INFO: inline-volume-tester-l8h2w started at 2023-01-16 06:34:11 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container csi-volume-tester ready: true, restart count 0
Jan 16 06:40:04.843: INFO: ebs-csi-node-8bhxv started at 2023-01-16 06:26:07 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:40:04.843: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:40:04.843: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:40:04.843: INFO: csi-mockplugin-attacher-0 started at 2023-01-16 06:39:45 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container csi-attacher ready: false, restart count 0
Jan 16 06:40:04.843: INFO: oidc-discovery-validator started at 2023-01-16 06:39:56 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container oidc-discovery-validator ready: true, restart count 0
Jan 16 06:40:04.843: INFO: liveness-470354a2-716e-4341-b41b-afc9c1378c49 started at 2023-01-16 06:35:01 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:04.843: INFO: Container agnhost-container ready: true, restart count 0
Jan 16 06:40:05.169: INFO:
Latency metrics for node i-0ac9adcf8852f0594
Jan 16 06:40:05.169: INFO:
Logging node info for node i-0b3e7c1b3ffabe4e1
Jan 16 06:40:05.235: INFO: Node Info: &Node{ObjectMeta:{i-0b3e7c1b3ffabe4e1 f1f7f96e-8f6d-4178-9a45-c16cedbd8510 17657 0 2023-01-16 06:25:59 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b3e7c1b3ffabe4e1 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-0b3e7c1b3ffabe4e1 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.59.231 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volumemode-1482":"i-0b3e7c1b3ffabe4e1","csi-mock-csi-mock-volumes-8770":"i-0b3e7c1b3ffabe4e1","ebs.csi.aws.com":"i-0b3e7c1b3ffabe4e1"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2023-01-16 06:39:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-16 06:40:02 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0b3e7c1b3ffabe4e1,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:38:23 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:38:23 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:38:23 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:38:23 +0000 UTC,LastTransitionTime:2023-01-16 06:26:13 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.231,},NodeAddress{Type:ExternalIP,Address:52.12.10.229,},NodeAddress{Type:InternalDNS,Address:i-0b3e7c1b3ffabe4e1.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b3e7c1b3ffabe4e1.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-12-10-229.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec26e5d5e832a48110ff0427b389577a,SystemUUID:ec26e5d5-e832-a481-10ff-0427b389577a,BootID:21853215-8129-4e64-b14a-211cf0ee9b99,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d49f269edca5f14a],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d49f269edca5f14a,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-8770^9a554817-9568-11ed-a9a1-96dc76e35dce,DevicePath:,},},Config:nil,},}
Jan 16 06:40:05.235: INFO:
Logging kubelet events for node i-0b3e7c1b3ffabe4e1
Jan 16 06:40:05.303: INFO:
Logging pods the kubelet thinks is on node i-0b3e7c1b3ffabe4e1
Jan 16 06:40:05.408: INFO: cilium-l9bsn started at 2023-01-16 06:26:00 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:40:05.408: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:40:05.408: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:40:05.408: INFO: csi-mockplugin-0 started at 2023-01-16 06:39:48 +0000 UTC (0+4 container statuses recorded)
Jan 16 06:40:05.408: INFO: Container busybox ready: true, restart count 0
Jan 16 06:40:05.408: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:40:05.408: INFO: Container driver-registrar ready: true, restart count 0
Jan 16 06:40:05.408: INFO: Container mock ready: true, restart count 0
Jan 16 06:40:05.408: INFO: csi-mockplugin-0 started at 2023-01-16 06:38:22 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:40:05.408: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:40:05.408: INFO: Container driver-registrar ready: true, restart count 0
Jan 16 06:40:05.408: INFO: Container mock ready: true, restart count 0
Jan 16 06:40:05.408: INFO: csi-mockplugin-attacher-0 started at 2023-01-16 06:38:22 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:05.408: INFO: Container csi-attacher ready: true, restart count 0
Jan 16 06:40:05.408: INFO: netserver-3 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:05.408: INFO: Container webserver ready: false, restart count 0
Jan 16 06:40:05.408: INFO: rs-zpzfh started at 2023-01-16 06:39:50 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:05.408: INFO: Container donothing ready: false, restart count 0
Jan 16 06:40:05.408: INFO: ebs-csi-node-zg8kr started at 2023-01-16 06:26:00 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:40:05.408: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:40:05.408: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:40:05.408: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:40:05.408: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:38:12 +0000 UTC (0+7 container statuses recorded)
Jan 16 06:40:05.408: INFO: Container csi-attacher ready: true, restart count 0
Jan 16 06:40:05.408: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:40:05.408: INFO: Container csi-resizer ready: true, restart count 0
Jan 16 06:40:05.408: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 16 06:40:05.408: INFO: Container hostpath ready: true, restart count 0
Jan 16 06:40:05.408: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:40:05.408: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:40:05.408: INFO: rs-5bvsz started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:05.408: INFO: Container donothing ready: true, restart count 0
Jan 16 06:40:05.408: INFO: pvc-volume-tester-7s9fv started at <nil> (0+0 container statuses recorded)
Jan 16 06:40:05.408: INFO: inline-volume-tester-25k5d started at 2023-01-16 06:33:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:05.408: INFO: Container csi-volume-tester ready: true, restart count 0
Jan 16 06:40:05.408: INFO: coredns-559769c974-v2nv5 started at 2023-01-16 06:26:26 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:05.408: INFO: Container coredns ready: true, restart count 0
Jan 16 06:40:05.713: INFO:
Latency metrics for node i-0b3e7c1b3ffabe4e1
Jan 16 06:40:05.713: INFO:
Logging node info for node i-0bcdb98d646a60a68
Jan 16 06:40:05.782: INFO: Node Info: &Node{ObjectMeta:{i-0bcdb98d646a60a68 ef8446c9-df03-44e4-bfed-57c522916505 14924 0 2023-01-16 06:24:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0bcdb98d646a60a68 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.49.7 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bcdb98d646a60a68"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-16 06:24:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-16 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:24:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:24:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:24:54 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-01-16 06:36:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0bcdb98d646a60a68,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.49.7,},NodeAddress{Type:ExternalIP,Address:35.164.29.62,},NodeAddress{Type:InternalDNS,Address:i-0bcdb98d646a60a68.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0bcdb98d646a60a68.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-164-29-62.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2e02b7de327b42fbf146d4a7087a79,SystemUUID:ec2e02b7-de32-7b42-fbf1-46d4a7087a79,BootID:8ddb4053-2513-4e18-b49d-f322ee6d1f6e,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43640493,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42802003,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967313,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 16 06:40:05.783: INFO:
Logging kubelet events for node i-0bcdb98d646a60a68
Jan 16 06:40:05.851: INFO:
Logging pods the kubelet thinks is on node i-0bcdb98d646a60a68
Jan 16 06:40:05.924: INFO: kube-apiserver-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+2 container statuses recorded)
Jan 16 06:40:05.924: INFO: Container healthcheck ready: true, restart count 0
Jan 16 06:40:05.924: INFO: Container kube-apiserver ready: true, restart count 1
Jan 16 06:40:05.924: INFO: cilium-qwpgj started at 2023-01-16 06:24:42 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:40:05.924: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:40:05.924: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:40:05.924: INFO: kops-controller-bjgn2 started at 2023-01-16 06:24:42 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:05.924: INFO: Container kops-controller ready: true, restart count 0
Jan 16 06:40:05.924: INFO: aws-cloud-controller-manager-fb8z2 started at 2023-01-16 06:24:42 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:05.924: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 16 06:40:05.924: INFO: cilium-operator-5dd44dc49f-sftdz started at 2023-01-16 06:24:43 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:05.924: INFO: Container cilium-operator ready: true, restart count 0
Jan 16 06:40:05.924: INFO: etcd-manager-main-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:05.924: INFO: Container etcd-manager ready: true, restart count 0
Jan 16 06:40:05.924: INFO: kube-controller-manager-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:05.924: INFO: Container kube-controller-manager ready: true, restart count 2
Jan 16 06:40:05.924: INFO: kube-scheduler-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:05.924: INFO: Container kube-scheduler ready: true, restart count 0
Jan 16 06:40:05.924: INFO: ebs-csi-node-9sg8r started at 2023-01-16 06:24:42 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:40:05.924: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:40:05.924: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:40:05.924: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:40:05.924: INFO: dns-controller-69987775c6-8t6m5 started at 2023-01-16 06:24:43 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:05.924: INFO: Container dns-controller ready: true, restart count 0
Jan 16 06:40:05.924: INFO: ebs-csi-controller-5bd98b456f-54tj7 started at 2023-01-16 06:24:43 +0000 UTC (0+5 container statuses recorded)
Jan 16 06:40:05.924: INFO: Container csi-attacher ready: true, restart count 0
Jan 16 06:40:05.924: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:40:05.924: INFO: Container csi-resizer ready: true, restart count 0
Jan 16 06:40:05.924: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:40:05.924: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:40:05.924: INFO: etcd-manager-events-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:40:05.924: INFO: Container etcd-manager ready: true, restart count 0
Jan 16 06:40:06.184: INFO:
Latency metrics for node i-0bcdb98d646a60a68
[DeferCleanup (Each)] [sig-network] Networking
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "nettest-5446" for this suite. �[38;5;243m01/16/23 06:40:06.184�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-node\]\skubelet\sClean\sup\spods\son\snode\skubelet\sshould\sbe\sable\sto\sdelete\s10\spods\sper\snode\sin\s1m0s\.$'
test/e2e/node/kubelet.go:359 k8s.io/kubernetes/test/e2e/node.glob..func5.2.3() test/e2e/node/kubelet.go:359 +0x390from junit_01.xml
[BeforeEach] [sig-node] kubelet set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/16/23 06:32:12.185�[0m Jan 16 06:32:12.186: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename kubelet �[38;5;243m01/16/23 06:32:12.186�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/16/23 06:32:12.383�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/16/23 06:32:12.514�[0m [BeforeEach] [sig-node] kubelet test/e2e/framework/metrics/init/init.go:31 [BeforeEach] [sig-node] kubelet test/e2e/node/kubelet.go:278 [BeforeEach] Clean up pods on node test/e2e/node/kubelet.go:299 [It] kubelet should be able to delete 10 pods per node in 1m0s. test/e2e/node/kubelet.go:346 �[1mSTEP:�[0m Creating a RC of 40 pods and wait until all pods of this RC are running �[38;5;243m01/16/23 06:32:13.131�[0m �[1mSTEP:�[0m creating replication controller cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 in namespace kubelet-9449 �[38;5;243m01/16/23 06:32:13.131�[0m I0116 06:32:13.198774 6597 runners.go:193] Created replication controller with name: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09, namespace: kubelet-9449, replica count: 40 Jan 16 06:32:13.277: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:32:13.357: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:32:13.592: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:32:13.685: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:32:13.759: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:32:18.402: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:32:18.481: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:32:18.710: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:32:18.864: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:32:18.870: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:32:23.299499 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 11 running, 29 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:32:23.503: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:32:23.589: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:32:23.820: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:32:23.966: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:32:23.998: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:32:28.671: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:32:28.759: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:32:28.947: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:32:29.069: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:32:29.156: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:32:33.300298 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 28 running, 12 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:32:33.796: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:32:33.871: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:32:34.140: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:32:34.198: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:32:34.291: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:32:38.904: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:32:38.974: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:32:39.253: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:32:39.360: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:32:39.471: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:32:43.301292 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 30 running, 10 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:32:44.023: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:32:44.100: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:32:44.431: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:32:44.490: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:32:44.641: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:32:49.148: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:32:49.206: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:32:49.592: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:32:49.609: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:32:49.778: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:32:53.302345 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 33 running, 7 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:32:54.261: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:32:54.335: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:32:54.712: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:32:54.736: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:32:54.892: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:32:59.373: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:32:59.458: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:32:59.830: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:32:59.867: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:33:00.012: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:33:03.303509 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 33 running, 7 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:33:04.474: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:33:04.600: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:33:04.957: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:33:04.994: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:33:05.139: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:33:09.583: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:33:09.790: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:33:10.098: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:33:10.204: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:33:10.468: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:33:13.303942 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 35 running, 5 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:33:14.686: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:33:14.897: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:33:15.218: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:33:15.313: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:33:15.605: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:33:19.792: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:33:20.026: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:33:20.336: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:33:20.435: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:33:20.731: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:33:23.304201 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 35 running, 5 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:33:24.896: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:33:25.130: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:33:25.452: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:33:25.541: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:33:25.884: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:33:29.999: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:33:30.238: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:33:30.567: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:33:30.661: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:33:31.096: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:33:33.305260 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 35 running, 5 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:33:35.099: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:33:35.346: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:33:35.678: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:33:35.783: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:33:36.219: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:33:40.198: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:33:40.469: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:33:40.884: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:33:40.898: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:33:41.342: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:33:43.306347 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 35 running, 5 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:33:45.297: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:33:45.574: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:33:46.001: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:33:46.020: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:33:46.476: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:33:50.397: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:33:50.710: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:33:51.126: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:33:51.174: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:33:51.654: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:33:53.307238 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 35 running, 5 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:33:55.535: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:33:55.822: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:33:56.324: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:33:56.336: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:33:56.816: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:34:00.677: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:34:00.936: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:34:01.445: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:34:01.456: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:34:01.988: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:34:03.308114 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 36 running, 4 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:34:05.777: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:34:06.066: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:34:06.553: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:34:06.688: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:34:07.334: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:34:10.874: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:34:11.177: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:34:11.665: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:34:11.816: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:34:12.450: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:34:13.308707 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 36 running, 4 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:34:15.995: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:34:16.283: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:34:16.775: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:34:16.949: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:34:17.591: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:34:21.107: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:34:21.386: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:34:21.894: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:34:22.086: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:34:22.734: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:34:23.309752 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:34:26.209: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:34:26.500: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:34:26.997: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:34:27.207: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:34:27.834: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:34:31.323: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:34:31.605: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:34:32.101: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:34:32.332: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:34:33.075: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:34:33.310878 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:34:36.428: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:34:36.747: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:34:37.243: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:34:37.461: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:34:38.223: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:34:41.525: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:34:41.853: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:34:42.394: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:34:42.590: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" I0116 06:34:43.311703 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:34:43.329: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:34:46.623: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:34:46.954: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:34:47.503: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:34:47.715: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:34:48.468: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:34:51.723: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:34:52.060: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:34:52.611: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:34:52.835: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" I0116 06:34:53.312962 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:34:53.625: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:34:56.821: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:34:57.274: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:34:57.714: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:34:58.003: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:34:58.759: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:35:01.920: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:35:02.390: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:35:03.022: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:35:03.205: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" I0116 06:35:03.313204 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:35:03.901: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:35:07.024: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:35:07.555: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:35:08.139: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:35:08.318: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:35:09.022: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:35:12.120: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:35:12.654: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:35:13.262: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" I0116 06:35:13.313739 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:35:13.493: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:35:14.149: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:35:17.216: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:35:17.764: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:35:18.374: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:35:18.611: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:35:19.266: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:35:22.315: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:35:22.875: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" I0116 06:35:23.314348 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:35:23.506: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:35:23.727: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:35:24.383: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:35:27.415: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:35:27.994: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:35:28.615: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:35:28.871: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:35:29.496: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:35:32.513: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:35:33.105: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" I0116 06:35:33.314633 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:35:33.753: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:35:33.992: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:35:34.621: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:35:37.617: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:35:38.235: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:35:38.880: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:35:39.129: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:35:39.749: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:35:42.715: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" I0116 06:35:43.315077 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:35:43.343: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:35:44.001: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:35:44.244: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:35:44.865: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:35:47.813: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:35:48.446: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:35:49.120: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:35:49.361: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:35:49.993: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:35:52.912: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" I0116 06:35:53.315908 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:35:53.549: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:35:54.244: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:35:54.496: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:35:55.120: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:35:58.018: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:35:58.665: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:35:59.360: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:35:59.657: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:36:00.261: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:36:03.116: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" I0116 06:36:03.316539 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:36:03.775: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:36:04.503: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:36:04.762: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:36:05.386: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:36:08.217: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:36:08.896: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:36:09.610: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:36:09.917: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:36:10.495: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:36:13.316735 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:36:13.350: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:36:14.006: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:36:14.719: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:36:15.045: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:36:15.655: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:36:18.446: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:36:19.144: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:36:20.110: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:36:20.157: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:36:20.772: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:36:23.317008 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:36:23.550: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:36:24.249: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:36:25.224: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:36:25.292: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:36:25.961: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:36:28.647: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:36:29.358: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:36:30.332: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:36:30.432: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:36:31.149: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:36:33.318214 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:36:33.761: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:36:34.463: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:36:35.456: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:36:35.596: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:36:36.266: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:36:38.862: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:36:39.574: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:36:40.569: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:36:40.743: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:36:41.432: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:36:43.319249 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:36:43.970: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:36:44.679: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:36:45.735: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:36:45.872: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:36:46.550: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:36:49.058: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:36:49.843: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:36:50.948: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:36:50.992: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:36:51.686: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:36:53.320491 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:36:54.162: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:36:54.948: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:36:56.065: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:36:56.106: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:36:56.799: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:36:59.261: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:37:00.078: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:37:01.219: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:37:01.245: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:37:02.051: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:37:03.321297 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:37:04.362: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:37:05.198: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:37:06.336: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:37:06.380: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:37:07.252: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:37:09.467: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:37:10.406: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:37:11.450: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:37:11.510: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:37:12.364: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:37:13.322356 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:37:14.569: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:37:15.514: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:37:16.573: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:37:16.649: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:37:17.469: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:37:19.669: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:37:20.619: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:37:21.684: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:37:21.771: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:37:22.581: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:37:23.323178 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:37:24.777: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:37:25.756: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:37:26.801: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:37:26.890: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:37:27.706: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:37:29.877: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:37:30.925: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:37:31.940: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:37:32.011: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:37:32.864: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:37:33.323498 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:37:34.975: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:37:36.028: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:37:37.051: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:37:37.140: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:37:37.980: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:37:40.073: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:37:41.128: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:37:42.167: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:37:42.270: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:37:43.156: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" I0116 06:37:43.324014 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:37:45.168: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:37:46.268: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:37:47.275: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:37:47.388: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:37:48.343: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:37:50.268: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:37:51.374: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:37:52.391: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:37:52.517: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" I0116 06:37:53.324953 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:37:53.460: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:37:55.366: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:37:56.502: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:37:57.498: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:37:57.639: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:37:58.577: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:38:00.466: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:38:01.709: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:38:02.612: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:38:02.850: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" I0116 06:38:03.325240 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:38:03.688: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:38:05.591: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:38:06.832: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:38:07.742: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:38:08.010: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:38:08.875: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:38:10.691: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:38:11.950: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:38:12.874: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:38:13.139: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" I0116 06:38:13.326139 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:38:14.083: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:38:15.796: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:38:17.048: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:38:18.012: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:38:18.248: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:38:19.204: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:38:20.898: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:38:22.197: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:38:23.119: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" I0116 06:38:23.326470 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:38:23.373: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:38:24.315: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:38:26.001: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:38:27.297: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:38:28.226: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:38:28.497: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:38:29.427: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:38:31.104: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:38:32.408: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" I0116 06:38:33.327409 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:38:33.354: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:38:33.614: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:38:34.539: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:38:36.209: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:38:37.517: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:38:38.464: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:38:38.754: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:38:39.696: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:38:41.309: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:38:42.635: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" I0116 06:38:43.328363 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:38:43.574: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:38:43.874: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:38:44.866: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:38:46.410: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:38:47.746: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:38:48.744: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:38:48.982: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:38:49.971: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:38:51.511: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:38:52.860: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" I0116 06:38:53.328682 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:38:53.895: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:38:54.109: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:38:55.084: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:38:56.616: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:38:57.984: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:38:59.007: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:38:59.239: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:39:00.198: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:39:01.714: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:39:03.090: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" I0116 06:39:03.329019 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:39:04.165: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:39:04.359: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:39:05.307: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:39:06.991: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:39:08.236: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:39:09.272: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:39:09.479: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:39:10.439: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:39:12.091: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" I0116 06:39:13.330172 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 16 06:39:13.341: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:39:14.384: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:39:14.600: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:39:15.575: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:39:17.189: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" Jan 16 06:39:18.483: INFO: Missing info/stats for container "runtime" on node "i-0b3e7c1b3ffabe4e1" Jan 16 06:39:19.489: INFO: Missing info/stats for container "runtime" on node "i-09cdc5f20702b22f2" Jan 16 06:39:19.776: INFO: Missing info/stats for container "runtime" on node "i-09cbbd5e7917c52c5" Jan 16 06:39:20.706: INFO: Missing info/stats for container "runtime" on node "i-0ac9adcf8852f0594" Jan 16 06:39:22.285: INFO: Missing info/stats for container "runtime" on node "i-0bcdb98d646a60a68" I0116 06:39:23.330920 6597 runners.go:193] cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0116 06:39:23.398519 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2jtsn i-09cdc5f20702b22f2 Running <nil> I0116 06:39:23.398620 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2k7jz i-0ac9adcf8852f0594 Running <nil> I0116 06:39:23.398628 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-4nrmv i-0b3e7c1b3ffabe4e1 Running <nil> I0116 06:39:23.398636 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5656k i-0ac9adcf8852f0594 Running <nil> I0116 06:39:23.398643 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5d2hr i-09cdc5f20702b22f2 Running <nil> I0116 06:39:23.398650 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5dxgn i-0b3e7c1b3ffabe4e1 Running <nil> I0116 06:39:23.398657 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7tft7 i-09cdc5f20702b22f2 Running <nil> I0116 06:39:23.398664 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7xc54 i-0ac9adcf8852f0594 Running <nil> I0116 06:39:23.398672 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7z54m i-0b3e7c1b3ffabe4e1 Running <nil> I0116 06:39:23.398679 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5 i-0ac9adcf8852f0594 Pending <nil> I0116 06:39:23.398686 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8 i-0b3e7c1b3ffabe4e1 Pending <nil> I0116 06:39:23.398693 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-b9xdt i-0ac9adcf8852f0594 Running <nil> I0116 06:39:23.398700 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-blsx2 i-09cbbd5e7917c52c5 Running <nil> I0116 06:39:23.398706 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl i-09cbbd5e7917c52c5 Pending <nil> I0116 06:39:23.398713 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c9wj5 i-09cbbd5e7917c52c5 Running <nil> I0116 06:39:23.398720 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ccx9k i-0b3e7c1b3ffabe4e1 Running <nil> I0116 06:39:23.398726 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-crnvb i-0ac9adcf8852f0594 Running <nil> I0116 06:39:23.398733 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5 i-09cdc5f20702b22f2 Running <nil> I0116 06:39:23.398740 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-fw4ct i-0b3e7c1b3ffabe4e1 Running <nil> I0116 06:39:23.398746 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gk8f6 i-09cbbd5e7917c52c5 Running <nil> I0116 06:39:23.398753 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gqf9j i-0b3e7c1b3ffabe4e1 Running <nil> I0116 06:39:23.398759 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-hlz4f i-0ac9adcf8852f0594 Running <nil> I0116 06:39:23.398766 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5 i-09cdc5f20702b22f2 Running <nil> I0116 06:39:23.398773 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jh4ht i-09cdc5f20702b22f2 Running <nil> I0116 06:39:23.398783 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jmpsk i-09cbbd5e7917c52c5 Running <nil> I0116 06:39:23.398789 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf i-0b3e7c1b3ffabe4e1 Running <nil> I0116 06:39:23.398796 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-lbzbs i-0ac9adcf8852f0594 Running <nil> I0116 06:39:23.398804 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-m6w4z i-09cbbd5e7917c52c5 Running <nil> I0116 06:39:23.398810 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nnzrq i-0b3e7c1b3ffabe4e1 Running <nil> I0116 06:39:23.398817 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nwpjj i-09cbbd5e7917c52c5 Running <nil> I0116 06:39:23.398823 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ptdxh i-09cdc5f20702b22f2 Running <nil> I0116 06:39:23.398830 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-pwdrm i-09cbbd5e7917c52c5 Running <nil> I0116 06:39:23.398836 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s2299 i-0b3e7c1b3ffabe4e1 Running <nil> I0116 06:39:23.398843 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s5kgk i-09cdc5f20702b22f2 Running <nil> I0116 06:39:23.398850 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-vvtkb i-09cbbd5e7917c52c5 Running <nil> I0116 06:39:23.398857 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-w4v5z i-0ac9adcf8852f0594 Running <nil> I0116 06:39:23.398864 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-x7cbm i-09cdc5f20702b22f2 Running <nil> I0116 06:39:23.398870 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7 i-0ac9adcf8852f0594 Running <nil> I0116 06:39:23.398877 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-z5z6l i-09cdc5f20702b22f2 Running <nil> I0116 06:39:23.398883 6597 runners.go:193] Pod cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-zrtfg i-09cbbd5e7917c52c5 Running <nil> Jan 16 06:39:23.398: INFO: Unexpected error: <*errors.errorString | 0xc000ed5dc0>: { s: "only 37 pods started out of 40", } Jan 16 06:39:23.399: FAIL: only 37 pods started out of 40 Full Stack Trace k8s.io/kubernetes/test/e2e/node.glob..func5.2.3() test/e2e/node/kubelet.go:359 +0x390 [AfterEach] Clean up pods on node test/e2e/node/kubelet.go:330 �[1mSTEP:�[0m removing the label kubelet_cleanup off the node i-0b3e7c1b3ffabe4e1 �[38;5;243m01/16/23 06:39:23.399�[0m �[1mSTEP:�[0m verifying the node doesn't have the label kubelet_cleanup �[38;5;243m01/16/23 06:39:23.596�[0m �[1mSTEP:�[0m removing the label kubelet_cleanup off the node i-09cbbd5e7917c52c5 �[38;5;243m01/16/23 06:39:23.661�[0m �[1mSTEP:�[0m verifying the node doesn't have the label kubelet_cleanup �[38;5;243m01/16/23 06:39:23.797�[0m �[1mSTEP:�[0m removing the label kubelet_cleanup off the node i-09cdc5f20702b22f2 �[38;5;243m01/16/23 06:39:23.861�[0m �[1mSTEP:�[0m verifying the node doesn't have the label kubelet_cleanup �[38;5;243m01/16/23 06:39:23.997�[0m �[1mSTEP:�[0m removing the label kubelet_cleanup off the node i-0ac9adcf8852f0594 �[38;5;243m01/16/23 06:39:24.062�[0m �[1mSTEP:�[0m verifying the node doesn't have the label kubelet_cleanup �[38;5;243m01/16/23 06:39:24.199�[0m [AfterEach] [sig-node] kubelet test/e2e/framework/node/init/init.go:32 Jan 16 06:39:24.264: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-node] kubelet test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-node] kubelet dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/16/23 06:39:24.33�[0m �[1mSTEP:�[0m Collecting events from namespace "kubelet-9449". �[38;5;243m01/16/23 06:39:24.33�[0m �[1mSTEP:�[0m Found 223 events. �[38;5;243m01/16/23 06:39:24.402�[0m Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09: {replication-controller } SuccessfulCreate: Created pod: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7xc54 Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09: {replication-controller } SuccessfulCreate: Created pod: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2jtsn Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09: {replication-controller } SuccessfulCreate: Created pod: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-fw4ct Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09: {replication-controller } SuccessfulCreate: Created pod: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gk8f6 Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09: {replication-controller } SuccessfulCreate: Created pod: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7tft7 Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09: {replication-controller } SuccessfulCreate: Created pod: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ptdxh Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09: {replication-controller } SuccessfulCreate: Created pod: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gqf9j Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09: {replication-controller } SuccessfulCreate: Created pod: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-vvtkb Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09: {replication-controller } SuccessfulCreate: Created pod: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-lbzbs Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2jtsn: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2jtsn to i-09cdc5f20702b22f2 Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2k7jz: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2k7jz to i-0ac9adcf8852f0594 Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-4nrmv: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-4nrmv to i-0b3e7c1b3ffabe4e1 Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5656k: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5656k to i-0ac9adcf8852f0594 Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5d2hr: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5d2hr to i-09cdc5f20702b22f2 Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5dxgn: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5dxgn to i-0b3e7c1b3ffabe4e1 Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7tft7: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7tft7 to i-09cdc5f20702b22f2 Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7xc54: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7xc54 to i-0ac9adcf8852f0594 Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7z54m: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7z54m to i-0b3e7c1b3ffabe4e1 Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5 to i-0ac9adcf8852f0594 Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8 to i-0b3e7c1b3ffabe4e1 Jan 16 06:39:24.402: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-b9xdt: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-b9xdt to i-0ac9adcf8852f0594 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-blsx2: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-blsx2 to i-09cbbd5e7917c52c5 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl to i-09cbbd5e7917c52c5 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c9wj5: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c9wj5 to i-09cbbd5e7917c52c5 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-crnvb: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-crnvb to i-0ac9adcf8852f0594 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5 to i-09cdc5f20702b22f2 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-fw4ct: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-fw4ct to i-0b3e7c1b3ffabe4e1 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gk8f6: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gk8f6 to i-09cbbd5e7917c52c5 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gqf9j: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gqf9j to i-0b3e7c1b3ffabe4e1 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-hlz4f: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-hlz4f to i-0ac9adcf8852f0594 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5 to i-09cdc5f20702b22f2 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jh4ht: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jh4ht to i-09cdc5f20702b22f2 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jmpsk: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jmpsk to i-09cbbd5e7917c52c5 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf to i-0b3e7c1b3ffabe4e1 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-lbzbs: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-lbzbs to i-0ac9adcf8852f0594 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-m6w4z: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-m6w4z to i-09cbbd5e7917c52c5 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nnzrq: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nnzrq to i-0b3e7c1b3ffabe4e1 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nwpjj: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nwpjj to i-09cbbd5e7917c52c5 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ptdxh: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ptdxh to i-09cdc5f20702b22f2 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-pwdrm: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-pwdrm to i-09cbbd5e7917c52c5 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s2299: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s2299 to i-0b3e7c1b3ffabe4e1 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s5kgk: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s5kgk to i-09cdc5f20702b22f2 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-vvtkb: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-vvtkb to i-09cbbd5e7917c52c5 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-w4v5z: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-w4v5z to i-0ac9adcf8852f0594 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-x7cbm: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-x7cbm to i-09cdc5f20702b22f2 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7 to i-0ac9adcf8852f0594 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-z5z6l: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-z5z6l to i-09cdc5f20702b22f2 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:13 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-zrtfg: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-zrtfg to i-09cbbd5e7917c52c5 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:14 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09: {replication-controller } SuccessfulCreate: (combined from similar events): Created pod: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s5kgk Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:14 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ccx9k: {default-scheduler } Scheduled: Successfully assigned kubelet-9449/cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ccx9k to i-0b3e7c1b3ffabe4e1 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:14 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-fw4ct: {kubelet i-0b3e7c1b3ffabe4e1} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:14 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-fw4ct: {kubelet i-0b3e7c1b3ffabe4e1} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2jtsn: {kubelet i-09cdc5f20702b22f2} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2jtsn: {kubelet i-09cdc5f20702b22f2} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2jtsn: {kubelet i-09cdc5f20702b22f2} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5dxgn: {kubelet i-0b3e7c1b3ffabe4e1} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5dxgn: {kubelet i-0b3e7c1b3ffabe4e1} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7z54m: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1ace6d1e003a1be5f4d0eb2f2bbbfad90e9e19218af0f3abeb72dedde5f1d6a4": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4917f4040501e889f755742b6db52f89b94f86f43a924dcb19aac293c9c92521": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-blsx2: {kubelet i-09cbbd5e7917c52c5} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-blsx2: {kubelet i-09cbbd5e7917c52c5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-fw4ct: {kubelet i-0b3e7c1b3ffabe4e1} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gk8f6: {kubelet i-09cbbd5e7917c52c5} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gk8f6: {kubelet i-09cbbd5e7917c52c5} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gk8f6: {kubelet i-09cbbd5e7917c52c5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gqf9j: {kubelet i-0b3e7c1b3ffabe4e1} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gqf9j: {kubelet i-0b3e7c1b3ffabe4e1} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gqf9j: {kubelet i-0b3e7c1b3ffabe4e1} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "99c0888eb0aa3ae0a5905f8a29bb359ac9d5a4cc99b93854924eee2ee1cc4628": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nnzrq: {kubelet i-0b3e7c1b3ffabe4e1} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nnzrq: {kubelet i-0b3e7c1b3ffabe4e1} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ptdxh: {kubelet i-09cdc5f20702b22f2} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ptdxh: {kubelet i-09cdc5f20702b22f2} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-pwdrm: {kubelet i-09cbbd5e7917c52c5} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-pwdrm: {kubelet i-09cbbd5e7917c52c5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-pwdrm: {kubelet i-09cbbd5e7917c52c5} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-vvtkb: {kubelet i-09cbbd5e7917c52c5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-vvtkb: {kubelet i-09cbbd5e7917c52c5} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-vvtkb: {kubelet i-09cbbd5e7917c52c5} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-4nrmv: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "40b718e81903bf913ae57344a3c207c627ce3ecfaa22b27261b366c8a27b8e4a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5656k: {kubelet i-0ac9adcf8852f0594} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5656k: {kubelet i-0ac9adcf8852f0594} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5d2hr: {kubelet i-09cdc5f20702b22f2} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5d2hr: {kubelet i-09cdc5f20702b22f2} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5d2hr: {kubelet i-09cdc5f20702b22f2} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5dxgn: {kubelet i-0b3e7c1b3ffabe4e1} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7tft7: {kubelet i-09cdc5f20702b22f2} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7tft7: {kubelet i-09cdc5f20702b22f2} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7tft7: {kubelet i-09cdc5f20702b22f2} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7xc54: {kubelet i-0ac9adcf8852f0594} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7xc54: {kubelet i-0ac9adcf8852f0594} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0ee331cc272f864e90dc393a6fd40e06fcd12af0ee945da8dc52061526b95e78": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-b9xdt: {kubelet i-0ac9adcf8852f0594} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-b9xdt: {kubelet i-0ac9adcf8852f0594} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-blsx2: {kubelet i-09cbbd5e7917c52c5} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ccx9k: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9329a80c4813473a42a4422f6b0a856cc689a3db2ed55273643f78dcc8d9e8bd": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0a97f0eb97be9cdd503d5c687fcf8b8deeb064decd8a9c3c0e9d71b3cc8d5899": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-hlz4f: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d4229098c36ece1dbb492647b2b4807d0d1907fe40a4c1e222526714db6b0e9b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "666714ec2b41a989f5c0d8e36b4db3149199716af60f26a3722e8896e28cc445": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jh4ht: {kubelet i-09cdc5f20702b22f2} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jh4ht: {kubelet i-09cdc5f20702b22f2} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jh4ht: {kubelet i-09cdc5f20702b22f2} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-lbzbs: {kubelet i-0ac9adcf8852f0594} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-lbzbs: {kubelet i-0ac9adcf8852f0594} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-lbzbs: {kubelet i-0ac9adcf8852f0594} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nnzrq: {kubelet i-0b3e7c1b3ffabe4e1} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ptdxh: {kubelet i-09cdc5f20702b22f2} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s2299: {kubelet i-0b3e7c1b3ffabe4e1} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s2299: {kubelet i-0b3e7c1b3ffabe4e1} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s2299: {kubelet i-0b3e7c1b3ffabe4e1} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s5kgk: {kubelet i-09cdc5f20702b22f2} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s5kgk: {kubelet i-09cdc5f20702b22f2} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s5kgk: {kubelet i-09cdc5f20702b22f2} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:16 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-x7cbm: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "55b045e7dfc0265796e7219decdb9ccb66117a72cbcabef2bf27ac2e099a75c1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5656k: {kubelet i-0ac9adcf8852f0594} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7xc54: {kubelet i-0ac9adcf8852f0594} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-b9xdt: {kubelet i-0ac9adcf8852f0594} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-crnvb: {kubelet i-0ac9adcf8852f0594} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-crnvb: {kubelet i-0ac9adcf8852f0594} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-crnvb: {kubelet i-0ac9adcf8852f0594} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-m6w4z: {kubelet i-09cbbd5e7917c52c5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-m6w4z: {kubelet i-09cbbd5e7917c52c5} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-m6w4z: {kubelet i-09cbbd5e7917c52c5} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nwpjj: {kubelet i-09cbbd5e7917c52c5} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nwpjj: {kubelet i-09cbbd5e7917c52c5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-w4v5z: {kubelet i-0ac9adcf8852f0594} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-w4v5z: {kubelet i-0ac9adcf8852f0594} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-w4v5z: {kubelet i-0ac9adcf8852f0594} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f62f088df07cafc4cb2cdce8a80478b0d2781bc42c81809888a5e936203eaec6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-z5z6l: {kubelet i-09cdc5f20702b22f2} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-z5z6l: {kubelet i-09cdc5f20702b22f2} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-z5z6l: {kubelet i-09cdc5f20702b22f2} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-zrtfg: {kubelet i-09cbbd5e7917c52c5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-zrtfg: {kubelet i-09cbbd5e7917c52c5} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:17 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-zrtfg: {kubelet i-09cbbd5e7917c52c5} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:18 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2k7jz: {kubelet i-0ac9adcf8852f0594} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:18 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2k7jz: {kubelet i-0ac9adcf8852f0594} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:18 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2k7jz: {kubelet i-0ac9adcf8852f0594} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:18 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c9wj5: {kubelet i-09cbbd5e7917c52c5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:18 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c9wj5: {kubelet i-09cbbd5e7917c52c5} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:18 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c9wj5: {kubelet i-09cbbd5e7917c52c5} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:18 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jmpsk: {kubelet i-09cbbd5e7917c52c5} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:18 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jmpsk: {kubelet i-09cbbd5e7917c52c5} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:18 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jmpsk: {kubelet i-09cbbd5e7917c52c5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:18 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nwpjj: {kubelet i-09cbbd5e7917c52c5} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:19 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cfc9aff1aedbb053bc70aeb7f7df359f88d0df81e0992ea9b0187fd7d9419f52": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:28 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-x7cbm: {kubelet i-09cdc5f20702b22f2} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:28 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-x7cbm: {kubelet i-09cdc5f20702b22f2} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:28 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-x7cbm: {kubelet i-09cdc5f20702b22f2} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:29 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-4nrmv: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "2c8fdefee6b3b3bf2cf2fe4b2ea71658fc7da264a3222b9122c1187fb17890a1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:29 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7z54m: {kubelet i-0b3e7c1b3ffabe4e1} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:29 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7z54m: {kubelet i-0b3e7c1b3ffabe4e1} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:29 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7z54m: {kubelet i-0b3e7c1b3ffabe4e1} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:29 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "52604bf9ab83a5223f969630afdc8676ef53b7ca44acad7d3059fcea3d070a25": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:29 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1da8854c1198e2f783df8780b8da9c27acde156069b789b5fcfd378d69c9d441": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:29 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ccx9k: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "82aa34541edc1ebc5bf984d8d0a448a9ff88536c5bd21ce2ea5b912d3dd1e00c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:29 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8f13a6e00b86202b31463b242fbcc948e47485ea6baafc94d5dd848d0ac0632c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:30 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3c79a73d7a1c5c6b03629acb18075737c825b46f3a284a525a078133ae369edd": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:31 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-hlz4f: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "dfd1b9fd53f30b6cb4776da6e105845be6d236e4f03b287c31550be73c576fff": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:31 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a3ea0d6b30d4d36fbf2dc1e60a292ceede2047565c5a3fa74731234aeb7700f2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:31 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ef606177a7f314427d8b9e26985f86e603454cd7b3e16b0b16198b67a2ff9c05": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:33 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "86611062aa6d637c09faf882b6f10d558aa838151ea9e1217c93d11d5ad1bcf7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:40 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "609db776218ae98ca57f222807baa9e24b7fcbaeb6f63675ba2bda024b687b65": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:42 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a026f998dbaff6fa42b418d36cdf8a2e8da422751a9a2c39b1018c2310a4af5a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:42 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "982c4974b2279c8f4545360420469e8608030192d215039a5b58a2201890123d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:42 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b9cb7567aec47eb91fc314961fa763b8ae38d0e56068e664b8549641fff9254f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:43 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-4nrmv: {kubelet i-0b3e7c1b3ffabe4e1} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:43 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-4nrmv: {kubelet i-0b3e7c1b3ffabe4e1} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:43 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-4nrmv: {kubelet i-0b3e7c1b3ffabe4e1} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:43 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4a20b2ba38e65a2b8e242a481dff1629b5c2715fea7abfdd496bac14880d849c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:43 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4302b90fcfaee1161feeb304bffe135e0a9b51adaac383ae4bda3dbf718b1f8e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:45 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5dd07db8ca207e11ecbcbf6f349f9aae010ad8e215eab4b63501a170ce1e8ecd": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:45 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ccx9k: {kubelet i-0b3e7c1b3ffabe4e1} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:45 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ccx9k: {kubelet i-0b3e7c1b3ffabe4e1} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:45 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ccx9k: {kubelet i-0b3e7c1b3ffabe4e1} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:47 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-hlz4f: {kubelet i-0ac9adcf8852f0594} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:47 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-hlz4f: {kubelet i-0ac9adcf8852f0594} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:47 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-hlz4f: {kubelet i-0ac9adcf8852f0594} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:53 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7307a8c4d6d6c339f127c94d3a9ebd6637da9bb079720b7eaa26fa4395dcfab3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:54 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "464a716434f722b6f13166a7756a179827902197392c2a283f4143360fb0094b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:56 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9f4bde3cba7fb363109b059528507abf3dd4f904356145456a19683d36266a0a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:56 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f128f4821652d04ba592730a3fadb16039a6f0e1aa307bf46aeb49001b6554f8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:57 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5: {kubelet i-09cdc5f20702b22f2} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:57 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5: {kubelet i-09cdc5f20702b22f2} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:57 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5: {kubelet i-09cdc5f20702b22f2} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:58 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b65b10ed6b797d53fc830337850b314184a3a6352590e692f366a809f23e4a93": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:32:58 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d1c3106b1c8c1b6ba628a056ec62f48f0fdaf9d8368cfd2f6e73a60ad3cf0019": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:06 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "05d76036ab8a2510729c07550077f6fff6320cfca53d2afd70a47318b7d957f1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:07 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "731dc270b6092080813063b8b7c356b2fe86ee85f4134dd19ac1640529820290": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:08 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "db4db318c0ce93411ed6c75b5a0ffa6b3fae82992fbdca1a0a52052643772088": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:09 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "74ffef93624618fc9ef234a54537223fdf839d241e4057fdbabfce67080f8d54": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:10 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf: {kubelet i-0b3e7c1b3ffabe4e1} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:10 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf: {kubelet i-0b3e7c1b3ffabe4e1} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:10 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf: {kubelet i-0b3e7c1b3ffabe4e1} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:12 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "05e54552af70d69adc8a50020e81c90f0b2b45edc860e10c4530de397f1ed902": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:18 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "60d9bcdd62ce7a3bfdadf291eade1befe1d59594b670372a76613261a8199638": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:21 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e32d9d45cd6f0c2043deaf211d8ecd45c9b87dfd99c359f869f868a275aeebfa": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:22 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "579216d4a543ec951c250789caa2ec1cf83433575a0f79cb9f7893a8458b52d1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:24 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "68e7d5338844cac94e459c2538c5668a03ad3d916d742fb2fd29cb297781463b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:25 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "267973d391d7f4b4eb055717df3fc2f46f2606db011615170ea4f1fbc84663ac": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:32 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "db07940b1592df6cdc64c5e13113a7369943fe6dc2e7c4392046c3dd4d49549d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:32 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b3ac24b3e8b6e300fff74503689f0cbb147e33bf7c980d514b1a97cdf3ccdcaf": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:35 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bf83f5579fbf8157a23ba34c3083d11ebc32e0355fe55008d4ae127b0ec73b32": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:35 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9c9c9ce30c38e4c6e5742af604b90d465a2bc698abad3fa51fdde1f04bdb859e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:40 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b0c2bb10a2a859440bdfe5910ac6d88970d33db7daa907db6da8c023eddd1c25": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:44 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5: {kubelet i-09cdc5f20702b22f2} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5dffbc390cbebbf70b85264444c405723f1c36a9db3dce87087ec3489cfc28ff": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:46 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5acffaeb9979a999fccfca1d0d75cf9d74c1366f6aa907ddc923d49f1d26891f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:47 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c7e2e5cae7b1cd364bd2f6783222078139fc4598469db1dd0ccf3968c2881d99": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:50 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b0e7e4f5c731964b794e90f9249dd1a964366ebf9244d33550acc496d21cd665": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:33:55 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1ed09729f2bc7304d73522f45abe529f77001b2b42f555dea4e246310f63ebde": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:34:00 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "235e38bbfe8f8679c173939715a8648f077fd6d07011c7f1abccbcb306c4e432": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:34:01 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5: {kubelet i-09cdc5f20702b22f2} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:34:01 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5: {kubelet i-09cdc5f20702b22f2} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:34:01 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5: {kubelet i-09cdc5f20702b22f2} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:34:01 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6d3bd8d8a097d0d2910e5e96c97654e179bdbda3c8af1eb524d20cb2e4e2518b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:34:04 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ff635e9408c45b6c947d7d0210632944326a3399d8127ab94732f1989be91a26": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:34:08 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4c5f387a1b5854f2a4dc81d9e53e11f054dc29d72feb647a3558152d3372f3d1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:34:14 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7: {kubelet i-0ac9adcf8852f0594} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 16 06:39:24.403: INFO: At 2023-01-16 06:34:14 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7: {kubelet i-0ac9adcf8852f0594} Started: Started container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:34:14 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7: {kubelet i-0ac9adcf8852f0594} Created: Created container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 Jan 16 06:39:24.403: INFO: At 2023-01-16 06:34:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5: {kubelet i-0ac9adcf8852f0594} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cc34fd7864e908dc5fd0f7143cce388ae535f4253e8740660e04569d7ed9e274": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:34:15 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8: {kubelet i-0b3e7c1b3ffabe4e1} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8b904657c778c7d43c092a3a9c957ed513a78a303ea9f2113e568ef1a6b9febe": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.403: INFO: At 2023-01-16 06:34:20 +0000 UTC - event for cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl: {kubelet i-09cbbd5e7917c52c5} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1c976a400d2aa41775cbf7a00de4b58a8fa892b5021254343fd5640abc2eee03": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 16 06:39:24.475: INFO: POD NODE PHASE GRACE CONDITIONS Jan 16 06:39:24.475: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2jtsn i-09cdc5f20702b22f2 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:15 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:15 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.475: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2k7jz i-0ac9adcf8852f0594 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-4nrmv i-0b3e7c1b3ffabe4e1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:44 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:44 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5656k i-0ac9adcf8852f0594 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5d2hr i-09cdc5f20702b22f2 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5dxgn i-0b3e7c1b3ffabe4e1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7tft7 i-09cdc5f20702b22f2 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7xc54 i-0ac9adcf8852f0594 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7z54m i-0b3e7c1b3ffabe4e1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:30 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:30 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5 i-0ac9adcf8852f0594 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC ContainersNotReady containers with unready status: [cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC ContainersNotReady containers with unready status: [cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8 i-0b3e7c1b3ffabe4e1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC ContainersNotReady containers with unready status: [cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC ContainersNotReady containers with unready status: [cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-b9xdt i-0ac9adcf8852f0594 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-blsx2 i-09cbbd5e7917c52c5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:16 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:16 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl i-09cbbd5e7917c52c5 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC ContainersNotReady containers with unready status: [cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC ContainersNotReady containers with unready status: [cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c9wj5 i-09cbbd5e7917c52c5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ccx9k i-0b3e7c1b3ffabe4e1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:14 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:45 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:45 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:14 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-crnvb i-0ac9adcf8852f0594 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5 i-09cdc5f20702b22f2 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:34:01 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:34:01 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-fw4ct i-0b3e7c1b3ffabe4e1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:16 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:16 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gk8f6 i-09cbbd5e7917c52c5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:16 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:16 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gqf9j i-0b3e7c1b3ffabe4e1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:16 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:16 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-hlz4f i-0ac9adcf8852f0594 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:48 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:48 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5 i-09cdc5f20702b22f2 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:58 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:58 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jh4ht i-09cdc5f20702b22f2 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jmpsk i-09cbbd5e7917c52c5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf i-0b3e7c1b3ffabe4e1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:33:10 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:33:10 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-lbzbs i-0ac9adcf8852f0594 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:16 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:16 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-m6w4z i-09cbbd5e7917c52c5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nnzrq i-0b3e7c1b3ffabe4e1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nwpjj i-09cbbd5e7917c52c5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ptdxh i-09cdc5f20702b22f2 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-pwdrm i-09cbbd5e7917c52c5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:16 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:16 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s2299 i-0b3e7c1b3ffabe4e1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s5kgk i-09cdc5f20702b22f2 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-vvtkb i-09cbbd5e7917c52c5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:16 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:16 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-w4v5z i-0ac9adcf8852f0594 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:17 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-x7cbm i-09cdc5f20702b22f2 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:29 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:29 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7 i-0ac9adcf8852f0594 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:34:14 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:34:14 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-z5z6l i-09cdc5f20702b22f2 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-zrtfg i-09cbbd5e7917c52c5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-16 06:32:13 +0000 UTC }] Jan 16 06:39:24.476: INFO: Jan 16 06:39:24.883: INFO: Logging node info for node i-09cbbd5e7917c52c5 Jan 16 06:39:24.948: INFO: Node Info: &Node{ObjectMeta:{i-09cbbd5e7917c52c5 516d50c9-4cfe-4b03-9036-fb7eaefbf36d 16060 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cbbd5e7917c52c5 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cbbd5e7917c52c5 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.43.77 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-3020":"i-09cbbd5e7917c52c5","ebs.csi.aws.com":"i-09cbbd5e7917c52c5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:32:44 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:37:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cbbd5e7917c52c5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:26:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.77,},NodeAddress{Type:ExternalIP,Address:52.25.36.68,},NodeAddress{Type:InternalDNS,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-36-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2589b28acc26043d6f13727e02d530,SystemUUID:ec2589b2-8acc-2604-3d6f-13727e02d530,BootID:f8305071-d74c-44c0-a267-8cc84ea9ad7b,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-3020^9489448c-9567-11ed-b056-f2aad9c35bb6],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-3020^9489448c-9567-11ed-b056-f2aad9c35bb6,DevicePath:,},},Config:nil,},} Jan 16 06:39:24.948: INFO: Logging kubelet events for node i-09cbbd5e7917c52c5 Jan 16 06:39:25.015: INFO: Logging pods the kubelet thinks is on node i-09cbbd5e7917c52c5 Jan 16 06:39:25.089: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:32:40 +0000 UTC (0+7 container statuses recorded) Jan 16 06:39:25.089: INFO: Container csi-attacher ready: true, restart count 0 Jan 16 06:39:25.089: INFO: Container csi-provisioner ready: true, restart count 0 Jan 16 06:39:25.089: INFO: Container csi-resizer ready: true, restart count 0 Jan 16 06:39:25.089: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 16 06:39:25.089: INFO: Container hostpath ready: true, restart count 0 Jan 16 06:39:25.089: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:39:25.089: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:39:25.089: INFO: rs-9ttnx started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container donothing ready: false, restart count 0 Jan 16 06:39:25.089: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nwpjj started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.089: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-pwdrm started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.089: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c9wj5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.089: INFO: cilium-xxhgc started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:39:25.089: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:39:25.089: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-vvtkb started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.089: INFO: inline-volume-tester-5b6t4 started at 2023-01-16 06:32:43 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 16 06:39:25.089: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-blsx2 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.089: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gk8f6 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.089: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jmpsk started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.089: INFO: rs-qcmvk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container donothing ready: false, restart count 0 Jan 16 06:39:25.089: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-m6w4z started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.089: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-zrtfg started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.089: INFO: ebs-csi-node-smxsx started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded) Jan 16 06:39:25.089: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:39:25.089: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:39:25.089: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:39:25.089: INFO: coredns-autoscaler-7cb5c5b969-lt7sb started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container autoscaler ready: true, restart count 0 Jan 16 06:39:25.089: INFO: coredns-559769c974-bxl2p started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container coredns ready: true, restart count 0 Jan 16 06:39:25.089: INFO: hostexec-i-09cbbd5e7917c52c5-68zbp started at 2023-01-16 06:39:11 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container agnhost-container ready: true, restart count 0 Jan 16 06:39:25.089: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0 Jan 16 06:39:25.089: INFO: netserver-0 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.089: INFO: Container webserver ready: false, restart count 0 Jan 16 06:39:25.089: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:36:13 +0000 UTC (0+7 container statuses recorded) Jan 16 06:39:25.089: INFO: Container csi-attacher ready: false, restart count 0 Jan 16 06:39:25.089: INFO: Container csi-provisioner ready: false, restart count 0 Jan 16 06:39:25.089: INFO: Container csi-resizer ready: false, restart count 0 Jan 16 06:39:25.089: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 16 06:39:25.089: INFO: Container hostpath ready: false, restart count 0 Jan 16 06:39:25.089: INFO: Container liveness-probe ready: false, restart count 0 Jan 16 06:39:25.089: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 16 06:39:25.371: INFO: Latency metrics for node i-09cbbd5e7917c52c5 Jan 16 06:39:25.371: INFO: Logging node info for node i-09cdc5f20702b22f2 Jan 16 06:39:25.438: INFO: Node Info: &Node{ObjectMeta:{i-09cdc5f20702b22f2 684a930e-dce6-418d-b0e9-bf389ce1a6d1 16066 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cdc5f20702b22f2 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cdc5f20702b22f2 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.120 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cdc5f20702b22f2"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2023-01-16 06:37:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cdc5f20702b22f2,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.120,},NodeAddress{Type:ExternalIP,Address:52.25.16.21,},NodeAddress{Type:InternalDNS,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-16-21.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec20ae9de53dd52df737771e3cb612ee,SystemUUID:ec20ae9d-e53d-d52d-f737-771e3cb612ee,BootID:731f2d80-218c-42a6-8438-95aae6d5c60c,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 16 06:39:25.438: INFO: Logging kubelet events for node i-09cdc5f20702b22f2 Jan 16 06:39:25.505: INFO: Logging pods the kubelet thinks is on node i-09cdc5f20702b22f2 Jan 16 06:39:25.581: INFO: service-deletion-9hpkv started at 2023-01-16 06:35:33 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container service-deletion ready: false, restart count 0 Jan 16 06:39:25.581: INFO: pod-secrets-5bc87a87-80c1-4cdc-bedc-1ae4930ee9bc started at 2023-01-16 06:35:05 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container secret-volume-test ready: false, restart count 0 Jan 16 06:39:25.581: INFO: rs-qzzlk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container donothing ready: false, restart count 0 Jan 16 06:39:25.581: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.581: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-x7cbm started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.581: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-z5z6l started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.581: INFO: ebs-csi-node-gk7tk started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded) Jan 16 06:39:25.581: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:39:25.581: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:39:25.581: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:39:25.581: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2jtsn started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.581: INFO: all-succeed-pff5k started at 2023-01-16 06:33:56 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container c ready: false, restart count 0 Jan 16 06:39:25.581: INFO: hairpin started at 2023-01-16 06:34:50 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container agnhost-container ready: false, restart count 0 Jan 16 06:39:25.581: INFO: dns-test-164a87cf-8613-4195-9244-1bac7a28d7df started at 2023-01-16 06:39:10 +0000 UTC (0+3 container statuses recorded) Jan 16 06:39:25.581: INFO: Container jessie-querier ready: false, restart count 0 Jan 16 06:39:25.581: INFO: Container querier ready: false, restart count 0 Jan 16 06:39:25.581: INFO: Container webserver ready: false, restart count 0 Jan 16 06:39:25.581: INFO: all-pods-removed-hgfff started at 2023-01-16 06:33:12 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container c ready: false, restart count 0 Jan 16 06:39:25.581: INFO: rs-99bkg started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container donothing ready: true, restart count 0 Jan 16 06:39:25.581: INFO: netserver-1 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container webserver ready: false, restart count 0 Jan 16 06:39:25.581: INFO: all-succeed-n4z2d started at 2023-01-16 06:34:38 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container c ready: false, restart count 0 Jan 16 06:39:25.581: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5d2hr started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.581: INFO: implicit-root-uid started at 2023-01-16 06:38:39 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container implicit-root-uid ready: false, restart count 0 Jan 16 06:39:25.581: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ptdxh started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.581: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jh4ht started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.581: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s5kgk started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.581: INFO: all-pods-removed-6lcv7 started at 2023-01-16 06:33:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container c ready: true, restart count 0 Jan 16 06:39:25.581: INFO: hostexec-i-09cdc5f20702b22f2-6w69n started at 2023-01-16 06:34:18 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container agnhost-container ready: true, restart count 0 Jan 16 06:39:25.581: INFO: cilium-wp5d7 started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:39:25.581: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:39:25.581: INFO: rs-h9gkk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container donothing ready: false, restart count 0 Jan 16 06:39:25.581: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.581: INFO: all-succeed-wcs5x started at 2023-01-16 06:34:52 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container c ready: false, restart count 0 Jan 16 06:39:25.581: INFO: pod-771554b8-0ed1-4422-82d5-a0a4dc92ec52 started at 2023-01-16 06:34:29 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container write-pod ready: false, restart count 0 Jan 16 06:39:25.581: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7tft7 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:25.581: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:25.889: INFO: Latency metrics for node i-09cdc5f20702b22f2 Jan 16 06:39:25.889: INFO: Logging node info for node i-0ac9adcf8852f0594 Jan 16 06:39:25.954: INFO: Node Info: &Node{ObjectMeta:{i-0ac9adcf8852f0594 5c88bf26-14b3-4940-8136-fa11c560695e 16068 0 2023-01-16 06:26:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0ac9adcf8852f0594 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-0ac9adcf8852f0594 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.221 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-4883":"i-0ac9adcf8852f0594","ebs.csi.aws.com":"i-0ac9adcf8852f0594"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:34:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-16 06:37:57 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0ac9adcf8852f0594,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.221,},NodeAddress{Type:ExternalIP,Address:35.90.38.176,},NodeAddress{Type:InternalDNS,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-90-38-176.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2092b4f74388fb9645e6987d21584e,SystemUUID:ec2092b4-f743-88fb-9645-e6987d21584e,BootID:6319a23e-d063-4cef-b18e-f1906415b7a3,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-051cd447c54259a6e kubernetes.io/csi/ebs.csi.aws.com^vol-0c3cdffdafd49e4d9],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-051cd447c54259a6e,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0c3cdffdafd49e4d9,DevicePath:,},},Config:nil,},} Jan 16 06:39:25.954: INFO: Logging kubelet events for node i-0ac9adcf8852f0594 Jan 16 06:39:26.021: INFO: Logging pods the kubelet thinks is on node i-0ac9adcf8852f0594 Jan 16 06:39:26.096: INFO: netserver-2 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container webserver ready: true, restart count 0 Jan 16 06:39:26.096: INFO: rs-vxcbj started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container donothing ready: false, restart count 0 Jan 16 06:39:26.096: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7xc54 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.096: INFO: csi-mockplugin-0 started at 2023-01-16 06:33:16 +0000 UTC (0+3 container statuses recorded) Jan 16 06:39:26.096: INFO: Container csi-provisioner ready: true, restart count 0 Jan 16 06:39:26.096: INFO: Container driver-registrar ready: true, restart count 0 Jan 16 06:39:26.096: INFO: Container mock ready: true, restart count 0 Jan 16 06:39:26.096: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-w4v5z started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.096: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-hlz4f started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.096: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-crnvb started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.096: INFO: pod-secrets-ec267146-752d-4009-8f59-fd4a77e1e8fd started at 2023-01-16 06:38:49 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container secret-volume-test ready: false, restart count 0 Jan 16 06:39:26.096: INFO: csi-mockplugin-attacher-0 started at 2023-01-16 06:38:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container csi-attacher ready: false, restart count 0 Jan 16 06:39:26.096: INFO: rs-vwwhw started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container donothing ready: false, restart count 0 Jan 16 06:39:26.096: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-b9xdt started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.096: INFO: rs-7fgjp started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container donothing ready: true, restart count 0 Jan 16 06:39:26.096: INFO: pvc-volume-tester-48722 started at 2023-01-16 06:35:01 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container volume-tester ready: false, restart count 0 Jan 16 06:39:26.096: INFO: all-succeed-b2zfw started at 2023-01-16 06:33:56 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container c ready: false, restart count 0 Jan 16 06:39:26.096: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:34:54 +0000 UTC (0+7 container statuses recorded) Jan 16 06:39:26.096: INFO: Container csi-attacher ready: false, restart count 0 Jan 16 06:39:26.096: INFO: Container csi-provisioner ready: false, restart count 0 Jan 16 06:39:26.096: INFO: Container csi-resizer ready: false, restart count 0 Jan 16 06:39:26.096: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 16 06:39:26.096: INFO: Container hostpath ready: false, restart count 0 Jan 16 06:39:26.096: INFO: Container liveness-probe ready: false, restart count 0 Jan 16 06:39:26.096: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 16 06:39:26.096: INFO: csi-mockplugin-0 started at 2023-01-16 06:38:36 +0000 UTC (0+3 container statuses recorded) Jan 16 06:39:26.096: INFO: Container csi-provisioner ready: false, restart count 0 Jan 16 06:39:26.096: INFO: Container driver-registrar ready: false, restart count 0 Jan 16 06:39:26.096: INFO: Container mock ready: false, restart count 0 Jan 16 06:39:26.096: INFO: cilium-9tdp4 started at 2023-01-16 06:26:07 +0000 UTC (1+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:39:26.096: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:39:26.096: INFO: inline-volume-tester-l8h2w started at 2023-01-16 06:34:11 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 16 06:39:26.096: INFO: pod-logs-websocket-f7fd16d2-6330-41d5-bd34-4939ca38a8bc started at 2023-01-16 06:38:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container main ready: false, restart count 0 Jan 16 06:39:26.096: INFO: ebs-csi-node-8bhxv started at 2023-01-16 06:26:07 +0000 UTC (0+3 container statuses recorded) Jan 16 06:39:26.096: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:39:26.096: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:39:26.096: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:39:26.096: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-lbzbs started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.096: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5656k started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.096: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-xzjx7 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.096: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2k7jz started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.096: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-822t5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0 Jan 16 06:39:26.096: INFO: liveness-470354a2-716e-4341-b41b-afc9c1378c49 started at 2023-01-16 06:35:01 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container agnhost-container ready: false, restart count 0 Jan 16 06:39:26.096: INFO: downwardapi-volume-f3832215-334b-4118-b46d-440628e07a46 started at 2023-01-16 06:38:52 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.096: INFO: Container client-container ready: false, restart count 0 Jan 16 06:39:26.378: INFO: Latency metrics for node i-0ac9adcf8852f0594 Jan 16 06:39:26.378: INFO: Logging node info for node i-0b3e7c1b3ffabe4e1 Jan 16 06:39:26.443: INFO: Node Info: &Node{ObjectMeta:{i-0b3e7c1b3ffabe4e1 f1f7f96e-8f6d-4178-9a45-c16cedbd8510 16058 0 2023-01-16 06:25:59 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b3e7c1b3ffabe4e1 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-0b3e7c1b3ffabe4e1 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.59.231 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b3e7c1b3ffabe4e1"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:25:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:33:15 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:38:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0b3e7c1b3ffabe4e1,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:38:23 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:38:23 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:38:23 +0000 UTC,LastTransitionTime:2023-01-16 06:25:59 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:38:23 +0000 UTC,LastTransitionTime:2023-01-16 06:26:13 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.59.231,},NodeAddress{Type:ExternalIP,Address:52.12.10.229,},NodeAddress{Type:InternalDNS,Address:i-0b3e7c1b3ffabe4e1.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b3e7c1b3ffabe4e1.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-12-10-229.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec26e5d5e832a48110ff0427b389577a,SystemUUID:ec26e5d5-e832-a481-10ff-0427b389577a,BootID:21853215-8129-4e64-b14a-211cf0ee9b99,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d49f269edca5f14a],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d49f269edca5f14a,DevicePath:,},},Config:nil,},} Jan 16 06:39:26.443: INFO: Logging kubelet events for node i-0b3e7c1b3ffabe4e1 Jan 16 06:39:26.511: INFO: Logging pods the kubelet thinks is on node i-0b3e7c1b3ffabe4e1 Jan 16 06:39:26.583: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-4nrmv started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.583: INFO: inline-volume-tester-25k5d started at 2023-01-16 06:33:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 16 06:39:26.583: INFO: netserver-3 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container webserver ready: false, restart count 0 Jan 16 06:39:26.583: INFO: csi-mockplugin-attacher-0 started at 2023-01-16 06:38:22 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container csi-attacher ready: false, restart count 0 Jan 16 06:39:26.583: INFO: coredns-559769c974-v2nv5 started at 2023-01-16 06:26:26 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container coredns ready: true, restart count 0 Jan 16 06:39:26.583: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:38:12 +0000 UTC (0+7 container statuses recorded) Jan 16 06:39:26.583: INFO: Container csi-attacher ready: false, restart count 0 Jan 16 06:39:26.583: INFO: Container csi-provisioner ready: false, restart count 0 Jan 16 06:39:26.583: INFO: Container csi-resizer ready: false, restart count 0 Jan 16 06:39:26.583: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 16 06:39:26.583: INFO: Container hostpath ready: false, restart count 0 Jan 16 06:39:26.583: INFO: Container liveness-probe ready: false, restart count 0 Jan 16 06:39:26.583: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 16 06:39:26.583: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gqf9j started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.583: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nnzrq started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.583: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s2299 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.583: INFO: ebs-csi-node-zg8kr started at 2023-01-16 06:26:00 +0000 UTC (0+3 container statuses recorded) Jan 16 06:39:26.583: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:39:26.583: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:39:26.583: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:39:26.583: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ccx9k started at 2023-01-16 06:32:14 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.583: INFO: cilium-l9bsn started at 2023-01-16 06:26:00 +0000 UTC (1+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:39:26.583: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:39:26.583: INFO: rs-5bvsz started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container donothing ready: true, restart count 0 Jan 16 06:39:26.583: INFO: rs-x97fb started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container donothing ready: true, restart count 0 Jan 16 06:39:26.583: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-fw4ct started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.583: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-8lwd8 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0 Jan 16 06:39:26.583: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5dxgn started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.583: INFO: csi-mockplugin-0 started at 2023-01-16 06:38:22 +0000 UTC (0+3 container statuses recorded) Jan 16 06:39:26.583: INFO: Container csi-provisioner ready: false, restart count 0 Jan 16 06:39:26.583: INFO: Container driver-registrar ready: false, restart count 0 Jan 16 06:39:26.583: INFO: Container mock ready: false, restart count 0 Jan 16 06:39:26.583: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-k7jpf started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.583: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7z54m started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:26.583: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0 Jan 16 06:39:26.861: INFO: Latency metrics for node i-0b3e7c1b3ffabe4e1 Jan 16 06:39:26.861: INFO: Logging node info for node i-0bcdb98d646a60a68 Jan 16 06:39:26.927: INFO: Node Info: &Node{ObjectMeta:{i-0bcdb98d646a60a68 ef8446c9-df03-44e4-bfed-57c522916505 14924 0 2023-01-16 06:24:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0bcdb98d646a60a68 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:us-west-2a topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.49.7 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bcdb98d646a60a68"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-16 06:24:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-16 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:24:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:24:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:taints":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:24:54 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kubelet Update v1 2023-01-16 06:36:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0bcdb98d646a60a68,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862941696 0} {<nil>} 3772404Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758084096 0} {<nil>} 3670004Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:36:22 +0000 UTC,LastTransitionTime:2023-01-16 06:24:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.49.7,},NodeAddress{Type:ExternalIP,Address:35.164.29.62,},NodeAddress{Type:InternalDNS,Address:i-0bcdb98d646a60a68.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0bcdb98d646a60a68.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-164-29-62.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2e02b7de327b42fbf146d4a7087a79,SystemUUID:ec2e02b7-de32-7b42-fbf1-46d4a7087a79,BootID:8ddb4053-2513-4e18-b49d-f322ee6d1f6e,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43640493,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42802003,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967313,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 16 06:39:26.935: INFO: Logging kubelet events for node i-0bcdb98d646a60a68 Jan 16 06:39:27.003: INFO: Logging pods the kubelet thinks is on node i-0bcdb98d646a60a68 Jan 16 06:39:27.073: INFO: kube-apiserver-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+2 container statuses recorded) Jan 16 06:39:27.073: INFO: Container healthcheck ready: true, restart count 0 Jan 16 06:39:27.073: INFO: Container kube-apiserver ready: true, restart count 1 Jan 16 06:39:27.073: INFO: cilium-qwpgj started at 2023-01-16 06:24:42 +0000 UTC (1+1 container statuses recorded) Jan 16 06:39:27.073: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 16 06:39:27.073: INFO: Container cilium-agent ready: true, restart count 0 Jan 16 06:39:27.073: INFO: kops-controller-bjgn2 started at 2023-01-16 06:24:42 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:27.073: INFO: Container kops-controller ready: true, restart count 0 Jan 16 06:39:27.073: INFO: aws-cloud-controller-manager-fb8z2 started at 2023-01-16 06:24:42 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:27.073: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 16 06:39:27.073: INFO: cilium-operator-5dd44dc49f-sftdz started at 2023-01-16 06:24:43 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:27.073: INFO: Container cilium-operator ready: true, restart count 0 Jan 16 06:39:27.073: INFO: etcd-manager-main-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:27.073: INFO: Container etcd-manager ready: true, restart count 0 Jan 16 06:39:27.073: INFO: kube-controller-manager-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:27.073: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 16 06:39:27.073: INFO: kube-scheduler-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:27.073: INFO: Container kube-scheduler ready: true, restart count 0 Jan 16 06:39:27.073: INFO: ebs-csi-node-9sg8r started at 2023-01-16 06:24:42 +0000 UTC (0+3 container statuses recorded) Jan 16 06:39:27.073: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:39:27.073: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:39:27.073: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 16 06:39:27.073: INFO: dns-controller-69987775c6-8t6m5 started at 2023-01-16 06:24:43 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:27.073: INFO: Container dns-controller ready: true, restart count 0 Jan 16 06:39:27.073: INFO: ebs-csi-controller-5bd98b456f-54tj7 started at 2023-01-16 06:24:43 +0000 UTC (0+5 container statuses recorded) Jan 16 06:39:27.073: INFO: Container csi-attacher ready: true, restart count 0 Jan 16 06:39:27.073: INFO: Container csi-provisioner ready: true, restart count 0 Jan 16 06:39:27.073: INFO: Container csi-resizer ready: true, restart count 0 Jan 16 06:39:27.073: INFO: Container ebs-plugin ready: true, restart count 0 Jan 16 06:39:27.073: INFO: Container liveness-probe ready: true, restart count 0 Jan 16 06:39:27.073: INFO: etcd-manager-events-i-0bcdb98d646a60a68 started at 2023-01-16 06:23:36 +0000 UTC (0+1 container statuses recorded) Jan 16 06:39:27.073: INFO: Container etcd-manager ready: true, restart count 0 Jan 16 06:39:27.340: INFO: Latency metrics for node i-0bcdb98d646a60a68 [DeferCleanup (Each)] [sig-node] kubelet tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "kubelet-9449" for this suite. �[38;5;243m01/16/23 06:39:27.34�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-storage\]\sCSI\smock\svolume\sCSIStorageCapacity\sCSIStorageCapacity\sused\,\sinsufficient\scapacity$'
test/e2e/storage/csi_mock_volume.go:201
k8s.io/kubernetes/test/e2e/storage.glob..func2.1({0x0, 0x0, 0x1, 0x1, 0x0, 0x0, 0xc000b634f0, {0xc0032209f0, 0x2f}, 0x0, ...})
test/e2e/storage/csi_mock_volume.go:201 +0x81c
k8s.io/kubernetes/test/e2e/storage.glob..func2.16.1()
test/e2e/storage/csi_mock_volume.go:1415 +0x164
[BeforeEach] [sig-storage] CSI mock volume
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/16/23 06:32:42.5�[0m
Jan 16 06:32:42.500: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename csi-mock-volumes �[38;5;243m01/16/23 06:32:42.501�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/16/23 06:32:42.692�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/16/23 06:32:42.817�[0m
[BeforeEach] [sig-storage] CSI mock volume
test/e2e/framework/metrics/init/init.go:31
[It] CSIStorageCapacity used, insufficient capacity
test/e2e/storage/csi_mock_volume.go:1413
�[1mSTEP:�[0m Building a driver namespace object, basename csi-mock-volumes-6664 �[38;5;243m01/16/23 06:32:42.957�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/16/23 06:32:43.211�[0m
�[1mSTEP:�[0m deploying csi mock driver �[38;5;243m01/16/23 06:32:43.338�[0m
Jan 16 06:32:43.598: INFO: creating *v1.ServiceAccount: csi-mock-volumes-6664-4693/csi-attacher
Jan 16 06:32:43.662: INFO: creating *v1.ClusterRole: external-attacher-runner-csi-mock-volumes-6664
Jan 16 06:32:43.662: INFO: Define cluster role external-attacher-runner-csi-mock-volumes-6664
Jan 16 06:32:43.726: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-csi-mock-volumes-6664
Jan 16 06:32:43.791: INFO: creating *v1.Role: csi-mock-volumes-6664-4693/external-attacher-cfg-csi-mock-volumes-6664
Jan 16 06:32:43.855: INFO: creating *v1.RoleBinding: csi-mock-volumes-6664-4693/csi-attacher-role-cfg
Jan 16 06:32:43.919: INFO: creating *v1.ServiceAccount: csi-mock-volumes-6664-4693/csi-provisioner
Jan 16 06:32:43.984: INFO: creating *v1.ClusterRole: external-provisioner-runner-csi-mock-volumes-6664
Jan 16 06:32:43.984: INFO: Define cluster role external-provisioner-runner-csi-mock-volumes-6664
Jan 16 06:32:44.048: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-csi-mock-volumes-6664
Jan 16 06:32:44.111: INFO: creating *v1.Role: csi-mock-volumes-6664-4693/external-provisioner-cfg-csi-mock-volumes-6664
Jan 16 06:32:44.180: INFO: creating *v1.RoleBinding: csi-mock-volumes-6664-4693/csi-provisioner-role-cfg
Jan 16 06:32:44.245: INFO: creating *v1.ServiceAccount: csi-mock-volumes-6664-4693/csi-resizer
Jan 16 06:32:44.308: INFO: creating *v1.ClusterRole: external-resizer-runner-csi-mock-volumes-6664
Jan 16 06:32:44.308: INFO: Define cluster role external-resizer-runner-csi-mock-volumes-6664
Jan 16 06:32:44.373: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-csi-mock-volumes-6664
Jan 16 06:32:44.438: INFO: creating *v1.Role: csi-mock-volumes-6664-4693/external-resizer-cfg-csi-mock-volumes-6664
Jan 16 06:32:44.502: INFO: creating *v1.RoleBinding: csi-mock-volumes-6664-4693/csi-resizer-role-cfg
Jan 16 06:32:44.566: INFO: creating *v1.ServiceAccount: csi-mock-volumes-6664-4693/csi-snapshotter
Jan 16 06:32:44.630: INFO: creating *v1.ClusterRole: external-snapshotter-runner-csi-mock-volumes-6664
Jan 16 06:32:44.630: INFO: Define cluster role external-snapshotter-runner-csi-mock-volumes-6664
Jan 16 06:32:44.695: INFO: creating *v1.ClusterRoleBinding: csi-snapshotter-role-csi-mock-volumes-6664
Jan 16 06:32:44.759: INFO: creating *v1.Role: csi-mock-volumes-6664-4693/external-snapshotter-leaderelection-csi-mock-volumes-6664
Jan 16 06:32:44.824: INFO: creating *v1.RoleBinding: csi-mock-volumes-6664-4693/external-snapshotter-leaderelection
Jan 16 06:32:44.889: INFO: creating *v1.ServiceAccount: csi-mock-volumes-6664-4693/csi-mock
Jan 16 06:32:44.954: INFO: creating *v1.ClusterRoleBinding: csi-controller-attacher-role-csi-mock-volumes-6664
Jan 16 06:32:45.018: INFO: creating *v1.ClusterRoleBinding: csi-controller-provisioner-role-csi-mock-volumes-6664
Jan 16 06:32:45.081: INFO: creating *v1.ClusterRoleBinding: csi-controller-cluster-driver-registrar-role-csi-mock-volumes-6664
Jan 16 06:32:45.145: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-csi-mock-volumes-6664
Jan 16 06:32:45.209: INFO: creating *v1.ClusterRoleBinding: csi-controller-resizer-role-csi-mock-volumes-6664
Jan 16 06:32:45.273: INFO: creating *v1.ClusterRoleBinding: csi-controller-snapshotter-role-csi-mock-volumes-6664
Jan 16 06:32:45.338: INFO: creating *v1.StorageClass: csi-mock-sc-csi-mock-volumes-6664
Jan 16 06:32:45.402: INFO: creating *v1.StatefulSet: csi-mock-volumes-6664-4693/csi-mockplugin
Jan 16 06:32:45.469: INFO: creating *v1.CSIDriver: csi-mock-csi-mock-volumes-6664
Jan 16 06:32:45.533: INFO: creating *v1.StatefulSet: csi-mock-volumes-6664-4693/csi-mockplugin-attacher
Jan 16 06:32:45.600: INFO: waiting up to 4m0s for CSIDriver "csi-mock-csi-mock-volumes-6664"
Jan 16 06:32:45.663: INFO: waiting for CSIDriver csi-mock-csi-mock-volumes-6664 to register on node i-09cbbd5e7917c52c5
Jan 16 06:38:28.422: INFO: Unexpected error: Failed to register CSIDriver csi-mock-csi-mock-volumes-6664:
<*errors.errorString | 0xc00125a3e0>: {
s: "error waiting for CSI driver csi-mock-csi-mock-volumes-6664 registration on node i-09cbbd5e7917c52c5: timed out waiting for the condition",
}
Jan 16 06:38:28.422: FAIL: Failed to register CSIDriver csi-mock-csi-mock-volumes-6664: error waiting for CSI driver csi-mock-csi-mock-volumes-6664 registration on node i-09cbbd5e7917c52c5: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage.glob..func2.1({0x0, 0x0, 0x1, 0x1, 0x0, 0x0, 0xc000b634f0, {0xc0032209f0, 0x2f}, 0x0, ...})
test/e2e/storage/csi_mock_volume.go:201 +0x81c
k8s.io/kubernetes/test/e2e/storage.glob..func2.16.1()
test/e2e/storage/csi_mock_volume.go:1415 +0x164
[AfterEach] [sig-storage] CSI mock volume
test/e2e/framework/node/init/init.go:32
Jan 16 06:38:28.423: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] CSIStorageCapacity
test/e2e/storage/drivers/csi.go:699
�[1mSTEP:�[0m deleting the test namespace: csi-mock-volumes-6664 �[38;5;243m01/16/23 06:38:28.487�[0m
�[1mSTEP:�[0m Collecting events from namespace "csi-mock-volumes-6664". �[38;5;243m01/16/23 06:38:28.487�[0m
�[1mSTEP:�[0m Found 0 events. �[38;5;243m01/16/23 06:38:28.55�[0m
Jan 16 06:38:28.613: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 16 06:38:28.613: INFO:
Jan 16 06:38:28.677: INFO:
Logging node info for node i-09cbbd5e7917c52c5
Jan 16 06:38:28.740: INFO: Node Info: &Node{ObjectMeta:{i-09cbbd5e7917c52c5 516d50c9-4cfe-4b03-9036-fb7eaefbf36d 15320 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cbbd5e7917c52c5 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cbbd5e7917c52c5 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.43.77 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-3020":"i-09cbbd5e7917c52c5","ebs.csi.aws.com":"i-09cbbd5e7917c52c5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:32:44 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-16 06:37:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cbbd5e7917c52c5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:37:56 +0000 UTC,LastTransitionTime:2023-01-16 06:26:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.43.77,},NodeAddress{Type:ExternalIP,Address:52.25.36.68,},NodeAddress{Type:InternalDNS,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cbbd5e7917c52c5.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-36-68.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2589b28acc26043d6f13727e02d530,SystemUUID:ec2589b2-8acc-2604-3d6f-13727e02d530,BootID:f8305071-d74c-44c0-a267-8cc84ea9ad7b,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-3020^9489448c-9567-11ed-b056-f2aad9c35bb6],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-3020^9489448c-9567-11ed-b056-f2aad9c35bb6,DevicePath:,},},Config:nil,},}
Jan 16 06:38:28.741: INFO:
Logging kubelet events for node i-09cbbd5e7917c52c5
Jan 16 06:38:28.806: INFO:
Logging pods the kubelet thinks is on node i-09cbbd5e7917c52c5
Jan 16 06:38:28.939: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-m6w4z started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:28.939: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-zrtfg started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:28.939: INFO: ebs-csi-node-smxsx started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:38:28.939: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:38:28.939: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:38:28.939: INFO: coredns-autoscaler-7cb5c5b969-lt7sb started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container autoscaler ready: true, restart count 0
Jan 16 06:38:28.939: INFO: coredns-559769c974-bxl2p started at 2023-01-16 06:26:04 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container coredns ready: true, restart count 0
Jan 16 06:38:28.939: INFO: csi-mockplugin-0 started at 2023-01-16 06:32:45 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container csi-provisioner ready: false, restart count 0
Jan 16 06:38:28.939: INFO: Container driver-registrar ready: false, restart count 0
Jan 16 06:38:28.939: INFO: Container mock ready: false, restart count 0
Jan 16 06:38:28.939: INFO: csi-mockplugin-attacher-0 started at 2023-01-16 06:32:45 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container csi-attacher ready: false, restart count 0
Jan 16 06:38:28.939: INFO: rs-qcmvk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container donothing ready: false, restart count 0
Jan 16 06:38:28.939: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c5tgl started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: false, restart count 0
Jan 16 06:38:28.939: INFO: netserver-0 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container webserver ready: false, restart count 0
Jan 16 06:38:28.939: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:36:13 +0000 UTC (0+7 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container csi-attacher ready: false, restart count 0
Jan 16 06:38:28.939: INFO: Container csi-provisioner ready: false, restart count 0
Jan 16 06:38:28.939: INFO: Container csi-resizer ready: false, restart count 0
Jan 16 06:38:28.939: INFO: Container csi-snapshotter ready: false, restart count 0
Jan 16 06:38:28.939: INFO: Container hostpath ready: false, restart count 0
Jan 16 06:38:28.939: INFO: Container liveness-probe ready: false, restart count 0
Jan 16 06:38:28.939: INFO: Container node-driver-registrar ready: false, restart count 0
Jan 16 06:38:28.939: INFO: csi-hostpathplugin-0 started at 2023-01-16 06:32:40 +0000 UTC (0+7 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container csi-attacher ready: true, restart count 0
Jan 16 06:38:28.939: INFO: Container csi-provisioner ready: true, restart count 0
Jan 16 06:38:28.939: INFO: Container csi-resizer ready: true, restart count 0
Jan 16 06:38:28.939: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 16 06:38:28.939: INFO: Container hostpath ready: true, restart count 0
Jan 16 06:38:28.939: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:38:28.939: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:38:28.939: INFO: rs-9ttnx started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container donothing ready: false, restart count 0
Jan 16 06:38:28.939: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-nwpjj started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:28.939: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-c9wj5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:28.939: INFO: cilium-xxhgc started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:38:28.939: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:38:28.939: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-vvtkb started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:28.939: INFO: hostpath-symlink-prep-provisioning-9295 started at 2023-01-16 06:34:07 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container init-volume-provisioning-9295 ready: false, restart count 0
Jan 16 06:38:28.939: INFO: inline-volume-tester-5b6t4 started at 2023-01-16 06:32:43 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 16 06:38:28.939: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-blsx2 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:28.939: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-pwdrm started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:28.939: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-gk8f6 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:28.939: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jmpsk started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:28.939: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:29.219: INFO:
Latency metrics for node i-09cbbd5e7917c52c5
Jan 16 06:38:29.219: INFO:
Logging node info for node i-09cdc5f20702b22f2
Jan 16 06:38:29.283: INFO: Node Info: &Node{ObjectMeta:{i-09cdc5f20702b22f2 684a930e-dce6-418d-b0e9-bf389ce1a6d1 15164 0 2023-01-16 06:25:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-09cdc5f20702b22f2 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-09cdc5f20702b22f2 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.120 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-09cdc5f20702b22f2"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-16 06:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-16 06:37:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-09cdc5f20702b22f2,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:25:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:37:24 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.120,},NodeAddress{Type:ExternalIP,Address:52.25.16.21,},NodeAddress{Type:InternalDNS,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-09cdc5f20702b22f2.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-52-25-16-21.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec20ae9de53dd52df737771e3cb612ee,SystemUUID:ec20ae9d-e53d-d52d-f737-771e3cb612ee,BootID:731f2d80-218c-42a6-8438-95aae6d5c60c,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 16 06:38:29.283: INFO:
Logging kubelet events for node i-09cdc5f20702b22f2
Jan 16 06:38:29.348: INFO:
Logging pods the kubelet thinks is on node i-09cdc5f20702b22f2
Jan 16 06:38:29.420: INFO: rs-99bkg started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.420: INFO: Container donothing ready: true, restart count 0
Jan 16 06:38:29.420: INFO: successful-jobs-history-limit-27897515-25xjd started at 2023-01-16 06:35:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.420: INFO: Container c ready: false, restart count 0
Jan 16 06:38:29.420: INFO: netserver-1 started at 2023-01-16 06:35:02 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.420: INFO: Container webserver ready: false, restart count 0
Jan 16 06:38:29.420: INFO: all-succeed-n4z2d started at 2023-01-16 06:34:38 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container c ready: false, restart count 0
Jan 16 06:38:29.421: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-5d2hr started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:29.421: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-ptdxh started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:29.421: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jh4ht started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:29.421: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-s5kgk started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:29.421: INFO: all-pods-removed-6lcv7 started at 2023-01-16 06:33:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container c ready: true, restart count 0
Jan 16 06:38:29.421: INFO: hostexec-i-09cdc5f20702b22f2-6w69n started at 2023-01-16 06:34:18 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container agnhost-container ready: true, restart count 0
Jan 16 06:38:29.421: INFO: cilium-wp5d7 started at 2023-01-16 06:25:52 +0000 UTC (1+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 16 06:38:29.421: INFO: Container cilium-agent ready: true, restart count 0
Jan 16 06:38:29.421: INFO: rs-h9gkk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container donothing ready: false, restart count 0
Jan 16 06:38:29.421: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-f99w5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:29.421: INFO: all-succeed-wcs5x started at 2023-01-16 06:34:52 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container c ready: false, restart count 0
Jan 16 06:38:29.421: INFO: pod-771554b8-0ed1-4422-82d5-a0a4dc92ec52 started at 2023-01-16 06:34:29 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container write-pod ready: false, restart count 0
Jan 16 06:38:29.421: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-7tft7 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:29.421: INFO: service-deletion-9hpkv started at 2023-01-16 06:35:33 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container service-deletion ready: false, restart count 0
Jan 16 06:38:29.421: INFO: pod-secrets-5bc87a87-80c1-4cdc-bedc-1ae4930ee9bc started at 2023-01-16 06:35:05 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container secret-volume-test ready: false, restart count 0
Jan 16 06:38:29.421: INFO: successful-jobs-history-limit-27897516-zff4c started at 2023-01-16 06:36:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container c ready: false, restart count 0
Jan 16 06:38:29.421: INFO: rs-qzzlk started at 2023-01-16 06:33:03 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container donothing ready: false, restart count 0
Jan 16 06:38:29.421: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-jgpv5 started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:29.421: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-x7cbm started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:29.421: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-z5z6l started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:29.421: INFO: ebs-csi-node-gk7tk started at 2023-01-16 06:25:52 +0000 UTC (0+3 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container ebs-plugin ready: true, restart count 0
Jan 16 06:38:29.421: INFO: Container liveness-probe ready: true, restart count 0
Jan 16 06:38:29.421: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 16 06:38:29.421: INFO: successful-jobs-history-limit-27897513-n4m26 started at 2023-01-16 06:33:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container c ready: false, restart count 0
Jan 16 06:38:29.421: INFO: successful-jobs-history-limit-27897514-pt4qd started at 2023-01-16 06:34:00 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container c ready: false, restart count 0
Jan 16 06:38:29.421: INFO: cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09-2jtsn started at 2023-01-16 06:32:13 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container cleanup40-d0be5ecc-0060-4a0e-8a21-67176bc91b09 ready: true, restart count 0
Jan 16 06:38:29.421: INFO: all-succeed-pff5k started at 2023-01-16 06:33:56 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container c ready: false, restart count 0
Jan 16 06:38:29.421: INFO: hairpin started at 2023-01-16 06:34:50 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container agnhost-container ready: false, restart count 0
Jan 16 06:38:29.421: INFO: all-pods-removed-hgfff started at 2023-01-16 06:33:12 +0000 UTC (0+1 container statuses recorded)
Jan 16 06:38:29.421: INFO: Container c ready: false, restart count 0
Jan 16 06:38:29.680: INFO:
Latency metrics for node i-09cdc5f20702b22f2
Jan 16 06:38:29.680: INFO:
Logging node info for node i-0ac9adcf8852f0594
Jan 16 06:38:29.744: INFO: Node Info: &Node{ObjectMeta:{i-0ac9adcf8852f0594 5c88bf26-14b3-4940-8136-fa11c560695e 15325 0 2023-01-16 06:26:06 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:us-west-2 failure-domain.beta.kubernetes.io/zone:us-west-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0ac9adcf8852f0594 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:us-west-2a topology.hostpath.csi/node:i-0ac9adcf8852f0594 topology.kubernetes.io/region:us-west-2 topology.kubernetes.io/zone:us-west-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.38.221 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-4883":"i-0ac9adcf8852f0594","ebs.csi.aws.com":"i-0ac9adcf8852f0594"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-16 06:26:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-16 06:26:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {e2e.test Update v1 2023-01-16 06:32:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-16 06:34:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-16 06:37:57 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///us-west-2a/i-0ac9adcf8852f0594,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:06 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-16 06:34:47 +0000 UTC,LastTransitionTime:2023-01-16 06:26:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.38.221,},NodeAddress{Type:ExternalIP,Address:35.90.38.176,},NodeAddress{Type:InternalDNS,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ac9adcf8852f0594.us-west-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-90-38-176.us-west-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2092b4f74388fb9645e6987d21584e,SystemUUID:ec2092b4-f743-88fb-9645-e6987d21584e,BootID:6319a23e-d063-4cef-b18e-f1906415b7a3,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k