Result | FAILURE |
Tests | 18 failed / 870 succeeded |
Started | |
Elapsed | 39m33s |
Revision | master |
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(Always\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\svia\schgrp\sin\sfirst\spod\,\snew\spod\swith\sdifferent\sfsgroup\sapplied\sto\sthe\svolume\scontents$'
test/e2e/storage/testsuites/fsgroupchangepolicy.go:258 k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc0018763c0, 0xc005275e10, 0x1, {0x0, 0x0}, {0x0, 0x0}) test/e2e/storage/testsuites/fsgroupchangepolicy.go:258 +0x151 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() test/e2e/storage/testsuites/fsgroupchangepolicy.go:229 +0x298from junit_01.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/09/23 06:38:59.659�[0m Jan 9 06:38:59.659: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename fsgroupchangepolicy �[38;5;243m01/09/23 06:38:59.66�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/09/23 06:39:00.093�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/09/23 06:39:00.379�[0m [BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy test/e2e/framework/metrics/init/init.go:31 [It] (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with different fsgroup applied to the volume contents test/e2e/storage/testsuites/fsgroupchangepolicy.go:210 Jan 9 06:39:00.664: INFO: Creating resource for dynamic PV Jan 9 06:39:00.665: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP:�[0m creating a StorageClass fsgroupchangepolicy-5880-e2e-sctd922 �[38;5;243m01/09/23 06:39:00.665�[0m �[1mSTEP:�[0m creating a claim �[38;5;243m01/09/23 06:39:00.816�[0m Jan 9 06:39:00.816: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP:�[0m Creating Pod in namespace fsgroupchangepolicy-5880 with fsgroup 1000 �[38;5;243m01/09/23 06:39:01.102�[0m Jan 9 06:39:01.246: INFO: Waiting up to 5m0s for pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d" in namespace "fsgroupchangepolicy-5880" to be "running" Jan 9 06:39:01.387: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 141.392723ms Jan 9 06:39:03.532: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2.285765238s Jan 9 06:39:05.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4.283708208s Jan 9 06:39:07.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 6.283108949s Jan 9 06:39:09.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 8.283466807s Jan 9 06:39:11.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 10.284178362s Jan 9 06:39:13.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 12.284546136s Jan 9 06:39:15.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 14.283303003s Jan 9 06:39:17.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 16.283420713s Jan 9 06:39:19.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 18.284299709s Jan 9 06:39:21.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 20.28430989s Jan 9 06:39:23.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 22.284323752s Jan 9 06:39:25.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 24.283599643s Jan 9 06:39:27.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 26.283494212s Jan 9 06:39:29.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 28.284338202s Jan 9 06:39:31.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 30.283509844s Jan 9 06:39:33.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 32.283578784s Jan 9 06:39:35.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 34.283367931s Jan 9 06:39:37.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 36.283709432s Jan 9 06:39:39.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 38.283746034s Jan 9 06:39:41.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 40.283738913s Jan 9 06:39:43.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 42.28423045s Jan 9 06:39:45.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 44.283656416s Jan 9 06:39:47.531: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 46.284698902s Jan 9 06:39:49.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 48.283500474s Jan 9 06:39:51.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 50.283585182s Jan 9 06:39:53.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 52.284193553s Jan 9 06:39:55.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 54.283333732s Jan 9 06:39:57.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 56.283692016s Jan 9 06:39:59.533: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 58.287620167s Jan 9 06:40:01.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.283761398s Jan 9 06:40:03.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.283417646s Jan 9 06:40:05.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.28376743s Jan 9 06:40:07.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.284640367s Jan 9 06:40:09.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.284117679s Jan 9 06:40:11.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.283686213s Jan 9 06:40:13.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.284523539s Jan 9 06:40:15.531: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.285447869s Jan 9 06:40:17.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.284116661s Jan 9 06:40:19.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.284064953s Jan 9 06:40:21.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.283835941s Jan 9 06:40:23.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.283639229s Jan 9 06:40:25.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.283959958s Jan 9 06:40:27.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.283942847s Jan 9 06:40:29.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.283829566s Jan 9 06:40:31.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.284492421s Jan 9 06:40:33.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.283698754s Jan 9 06:40:35.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.283165986s Jan 9 06:40:37.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.28394345s Jan 9 06:40:39.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.283656824s Jan 9 06:40:41.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.284453602s Jan 9 06:40:43.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.283429534s Jan 9 06:40:45.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.283518662s Jan 9 06:40:47.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.284380623s Jan 9 06:40:49.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.284558797s Jan 9 06:40:51.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.283751884s Jan 9 06:40:53.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.283646112s Jan 9 06:40:55.531: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.285411789s Jan 9 06:40:57.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.283673903s Jan 9 06:40:59.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.283846752s Jan 9 06:41:01.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.284606937s Jan 9 06:41:03.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.284633622s Jan 9 06:41:05.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.283759217s Jan 9 06:41:07.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.28352988s Jan 9 06:41:09.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.284176439s Jan 9 06:41:11.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.283993704s Jan 9 06:41:13.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.28382322s Jan 9 06:41:15.535: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.288843974s Jan 9 06:41:17.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.283503157s Jan 9 06:41:19.533: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.286719767s Jan 9 06:41:21.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.283527981s Jan 9 06:41:23.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.28363587s Jan 9 06:41:25.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.283578184s Jan 9 06:41:27.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.283285664s Jan 9 06:41:29.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.283510678s Jan 9 06:41:31.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.283380214s Jan 9 06:41:33.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.284089367s Jan 9 06:41:35.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.284136943s Jan 9 06:41:37.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.283406267s Jan 9 06:41:39.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.284648352s Jan 9 06:41:41.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.283517738s Jan 9 06:41:43.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.284067164s Jan 9 06:41:45.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.283477399s Jan 9 06:41:47.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.283791528s Jan 9 06:41:49.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.283634096s Jan 9 06:41:51.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.284494696s Jan 9 06:41:53.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.284171778s Jan 9 06:41:55.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.283590151s Jan 9 06:41:57.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.283661978s Jan 9 06:41:59.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.283828232s Jan 9 06:42:01.531: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.284969196s Jan 9 06:42:03.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.283466838s Jan 9 06:42:05.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.284670405s Jan 9 06:42:07.531: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.284861896s Jan 9 06:42:09.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.283535403s Jan 9 06:42:11.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.284137593s Jan 9 06:42:13.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.283610372s Jan 9 06:42:15.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.283304108s Jan 9 06:42:17.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.283741138s Jan 9 06:42:19.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.283761418s Jan 9 06:42:21.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.284485088s Jan 9 06:42:23.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.284228105s Jan 9 06:42:25.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.283372291s Jan 9 06:42:27.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.283573337s Jan 9 06:42:29.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.28341506s Jan 9 06:42:31.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.2836939s Jan 9 06:42:33.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.283589399s Jan 9 06:42:35.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.283717992s Jan 9 06:42:37.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.283510442s Jan 9 06:42:39.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.283678321s Jan 9 06:42:41.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.284594932s Jan 9 06:42:43.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.283300227s Jan 9 06:42:45.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.283791427s Jan 9 06:42:47.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.28357855s Jan 9 06:42:49.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.283673175s Jan 9 06:42:51.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.284653467s Jan 9 06:42:53.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.284420186s Jan 9 06:42:55.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.283672899s Jan 9 06:42:57.531: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.284866206s Jan 9 06:42:59.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.283719641s Jan 9 06:43:01.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.283659701s Jan 9 06:43:03.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.283564032s Jan 9 06:43:05.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.28457933s Jan 9 06:43:07.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.283350954s Jan 9 06:43:09.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.283599569s Jan 9 06:43:11.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.28355319s Jan 9 06:43:13.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.284134867s Jan 9 06:43:15.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.283362237s Jan 9 06:43:17.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.283621408s Jan 9 06:43:19.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.283601005s Jan 9 06:43:21.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.283645881s Jan 9 06:43:23.531: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.284996121s Jan 9 06:43:25.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.283336815s Jan 9 06:43:27.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.283361362s Jan 9 06:43:29.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.283477574s Jan 9 06:43:31.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.283466459s Jan 9 06:43:33.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.284163429s Jan 9 06:43:35.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.283324148s Jan 9 06:43:37.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.283502088s Jan 9 06:43:39.531: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.285104837s Jan 9 06:43:41.535: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.289094401s Jan 9 06:43:43.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.284280547s Jan 9 06:43:45.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.284289297s Jan 9 06:43:47.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.283671223s Jan 9 06:43:49.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.283272373s Jan 9 06:43:51.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.283551455s Jan 9 06:43:53.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.283588045s Jan 9 06:43:55.557: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.31119744s Jan 9 06:43:57.529: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.283555099s Jan 9 06:43:59.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.284612756s Jan 9 06:44:01.530: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.283829225s Jan 9 06:44:01.672: INFO: Pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.425897429s Jan 9 06:44:01.672: INFO: Unexpected error: <*errors.errorString | 0xc001234ee0>: { s: "pod \"pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d\" is not Running: timed out while waiting for pod fsgroupchangepolicy-5880/pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d to be running", } Jan 9 06:44:01.672: FAIL: pod "pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d" is not Running: timed out while waiting for pod fsgroupchangepolicy-5880/pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d to be running Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc0018763c0, 0xc005275e10, 0x1, {0x0, 0x0}, {0x0, 0x0}) test/e2e/storage/testsuites/fsgroupchangepolicy.go:258 +0x151 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() test/e2e/storage/testsuites/fsgroupchangepolicy.go:229 +0x298 �[1mSTEP:�[0m Deleting pvc �[38;5;243m01/09/23 06:44:01.672�[0m Jan 9 06:44:01.956: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com9pqzh" Jan 9 06:44:02.101: INFO: Waiting up to 5m0s for PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 to get deleted Jan 9 06:44:02.243: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (141.9524ms) Jan 9 06:44:07.385: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (5.284650182s) Jan 9 06:44:12.528: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (10.42765928s) Jan 9 06:44:17.672: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (15.570928392s) Jan 9 06:44:22.814: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (20.713363296s) Jan 9 06:44:27.957: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (25.856255543s) Jan 9 06:44:33.101: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (31.000419783s) Jan 9 06:44:38.244: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (36.14368188s) Jan 9 06:44:43.387: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (41.285951596s) Jan 9 06:44:48.529: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (46.428668326s) Jan 9 06:44:53.672: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (51.571736742s) Jan 9 06:44:58.815: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (56.714195181s) Jan 9 06:45:03.957: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (1m1.856721261s) Jan 9 06:45:09.100: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (1m6.998954068s) Jan 9 06:45:14.243: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (1m12.142305656s) Jan 9 06:45:19.385: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (1m17.284637318s) Jan 9 06:45:24.527: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (1m22.426726951s) Jan 9 06:45:29.670: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (1m27.569351512s) Jan 9 06:45:34.813: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (1m32.711910186s) Jan 9 06:45:39.955: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (1m37.854516134s) Jan 9 06:45:45.098: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (1m42.997226518s) Jan 9 06:45:50.240: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (1m48.139691176s) Jan 9 06:45:55.382: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (1m53.281800848s) Jan 9 06:46:00.525: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (1m58.424793244s) Jan 9 06:46:05.668: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (2m3.567012693s) Jan 9 06:46:10.811: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (2m8.709916112s) Jan 9 06:46:15.959: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (2m13.858211013s) Jan 9 06:46:21.101: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (2m19.000775865s) Jan 9 06:46:26.244: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (2m24.143371848s) Jan 9 06:46:31.386: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (2m29.285735084s) Jan 9 06:46:36.529: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (2m34.428668619s) Jan 9 06:46:41.672: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (2m39.57134507s) Jan 9 06:46:46.814: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (2m44.713170148s) Jan 9 06:46:51.956: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (2m49.855571498s) Jan 9 06:46:57.099: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (2m54.998375269s) Jan 9 06:47:02.242: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (3m0.141367258s) Jan 9 06:47:07.384: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (3m5.283809715s) Jan 9 06:47:12.527: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (3m10.426572025s) Jan 9 06:47:17.670: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (3m15.569701892s) Jan 9 06:47:22.813: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (3m20.712416562s) Jan 9 06:47:27.955: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (3m25.854804678s) Jan 9 06:47:33.098: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (3m30.997297345s) Jan 9 06:47:38.240: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (3m36.139879506s) Jan 9 06:47:43.383: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (3m41.281937297s) Jan 9 06:47:48.527: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (3m46.426480917s) Jan 9 06:47:53.670: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (3m51.569723984s) Jan 9 06:47:58.812: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (3m56.7117364s) Jan 9 06:48:03.956: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (4m1.854933579s) Jan 9 06:48:09.098: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (4m6.997716493s) Jan 9 06:48:14.241: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (4m12.140399553s) Jan 9 06:48:19.384: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (4m17.283885162s) Jan 9 06:48:24.528: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (4m22.426981952s) Jan 9 06:48:29.670: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (4m27.569125703s) Jan 9 06:48:34.812: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (4m32.711628871s) Jan 9 06:48:39.954: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (4m37.853633753s) Jan 9 06:48:45.096: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (4m42.995875092s) Jan 9 06:48:50.239: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (4m48.138495887s) Jan 9 06:48:55.381: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (4m53.280545912s) Jan 9 06:49:00.524: INFO: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 found and phase=Bound (4m58.423393513s) �[1mSTEP:�[0m Deleting sc �[38;5;243m01/09/23 06:49:05.525�[0m Jan 9 06:49:05.669: INFO: Unexpected error: while cleanup resource: <errors.aggregate | len:1, cap:1>: [ <errors.aggregate | len:1, cap:1>[ <*fmt.wrapError | 0xc005078060>{ msg: "persistent Volume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 not deleted by dynamic provisioner: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 still exists within 5m0s", err: <*errors.errorString | 0xc001234020>{ s: "PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 still exists within 5m0s", }, }, ], ] Jan 9 06:49:05.669: FAIL: while cleanup resource: persistent Volume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 not deleted by dynamic provisioner: PersistentVolume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 still exists within 5m0s Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2() test/e2e/storage/testsuites/fsgroupchangepolicy.go:128 +0x14e panic({0x70eb680, 0xc000a72850}) /usr/local/go/src/runtime/panic.go:884 +0x212 k8s.io/kubernetes/test/e2e/framework.Fail({0xc0011f6160, 0xae}, {0xc001af9c30?, 0xc0011f6160?, 0xc001af9c58?}) test/e2e/framework/log.go:61 +0x145 k8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0x1, {0x7fa4200, 0xc001234ee0}, {0x0?, 0x2?, 0x7f243e293108?}) test/e2e/framework/expect.go:76 +0x267 k8s.io/kubernetes/test/e2e/framework.ExpectNoError(...) test/e2e/framework/expect.go:43 k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc0018763c0, 0xc005275e10, 0x1, {0x0, 0x0}, {0x0, 0x0}) test/e2e/storage/testsuites/fsgroupchangepolicy.go:258 +0x151 k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3() test/e2e/storage/testsuites/fsgroupchangepolicy.go:229 +0x298 [AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy test/e2e/framework/node/init/init.go:32 Jan 9 06:49:05.670: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/09/23 06:49:05.812�[0m �[1mSTEP:�[0m Collecting events from namespace "fsgroupchangepolicy-5880". �[38;5;243m01/09/23 06:49:05.813�[0m �[1mSTEP:�[0m Found 17 events. �[38;5;243m01/09/23 06:49:05.955�[0m Jan 9 06:49:05.955: INFO: At 2023-01-09 06:39:00 +0000 UTC - event for ebs.csi.aws.com9pqzh: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Jan 9 06:49:05.955: INFO: At 2023-01-09 06:39:01 +0000 UTC - event for ebs.csi.aws.com9pqzh: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Jan 9 06:49:05.955: INFO: At 2023-01-09 06:39:01 +0000 UTC - event for ebs.csi.aws.com9pqzh: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-zwk75_90295703-a18c-4cf0-8631-b29017505c1e } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-5880/ebs.csi.aws.com9pqzh" Jan 9 06:49:05.955: INFO: At 2023-01-09 06:39:04 +0000 UTC - event for ebs.csi.aws.com9pqzh: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-zwk75_90295703-a18c-4cf0-8631-b29017505c1e } ProvisioningSucceeded: Successfully provisioned volume pvc-8ed0de19-6fd0-4474-a424-64a0a8912651 Jan 9 06:49:05.955: INFO: At 2023-01-09 06:39:05 +0000 UTC - event for pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d: {default-scheduler } Scheduled: Successfully assigned fsgroupchangepolicy-5880/pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d to i-087c2326a312024ef Jan 9 06:49:05.955: INFO: At 2023-01-09 06:39:12 +0000 UTC - event for pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-8ed0de19-6fd0-4474-a424-64a0a8912651" Jan 9 06:49:05.955: INFO: At 2023-01-09 06:39:15 +0000 UTC - event for pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "411d936c0f4d1dda06a93b44e85df67d74506c8794696ffc1d179db9d78f39c9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:49:05.955: INFO: At 2023-01-09 06:39:28 +0000 UTC - event for pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "de4e6867eae9857f65608d3151d27b729403642adc1d6b7aeb10135b1c1b72d9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:49:05.955: INFO: At 2023-01-09 06:39:41 +0000 UTC - event for pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e6a8ce5e716cd5538ba6956d01983886c3109548d9b93d6d7f562e3605eb1924": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:49:05.955: INFO: At 2023-01-09 06:39:54 +0000 UTC - event for pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "dc2d36d100c2614a0a2f5b4524289fd8d3e5687e77fcf3f2f22f3d8ccb8a81d7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:49:05.955: INFO: At 2023-01-09 06:40:08 +0000 UTC - event for pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cb4c08361088d97c9c2bd8c8ab033bfbbf339e0834dc6291179637ed171b1292": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:49:05.955: INFO: At 2023-01-09 06:40:22 +0000 UTC - event for pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "98c4faeeb91970ad34f0b9047d94b0c34a9c4192c44a263237c2f344acad75d0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:49:05.955: INFO: At 2023-01-09 06:40:35 +0000 UTC - event for pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "18c8c51610ad61487e152b9d459335ad897f563c6a6d78754b6a429374a0b341": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:49:05.955: INFO: At 2023-01-09 06:40:46 +0000 UTC - event for pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "26548c497f13a86851061b862e6e444352035690fe4c93307937489e4bd60df2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:49:05.955: INFO: At 2023-01-09 06:40:58 +0000 UTC - event for pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "30282860e77501f61f00097e8419463eee75f939a39fa03cac2f558e9c02bb48": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:49:05.955: INFO: At 2023-01-09 06:41:12 +0000 UTC - event for pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b576a9b2bcf2e6eb90a13c17ca4d6025461b3b4488ba30540a216eaf9f9c1fff": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:49:05.955: INFO: At 2023-01-09 06:44:40 +0000 UTC - event for pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d: {kubelet i-087c2326a312024ef} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine Jan 9 06:49:06.097: INFO: POD NODE PHASE GRACE CONDITIONS Jan 9 06:49:06.097: INFO: pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d i-087c2326a312024ef Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:05 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:44:41 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:44:41 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:05 +0000 UTC }] Jan 9 06:49:06.097: INFO: Jan 9 06:49:06.397: INFO: Logging node info for node i-02b09d19b2096261b Jan 9 06:49:06.539: INFO: Node Info: &Node{ObjectMeta:{i-02b09d19b2096261b efa11f02-6d54-4902-bbeb-efecc1f02cd1 32880 0 2023-01-09 06:23:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-02b09d19b2096261b kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.12 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02b09d19b2096261b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-09 06:23:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-09 06:24:22 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-09 06:24:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-09 06:46:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-02b09d19b2096261b,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:46:43 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:46:43 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:46:43 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:46:43 +0000 UTC,LastTransitionTime:2023-01-09 06:24:43 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.12,},NodeAddress{Type:ExternalIP,Address:54.94.119.205,},NodeAddress{Type:InternalDNS,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-119-205.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2da51bd0e40b36975082ac186a369f,SystemUUID:ec2da51b-d0e4-0b36-9750-82ac186a369f,BootID:83891f80-d7aa-4da9-ab10-44706ab3b631,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43453930,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42803458,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967347,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:49:06.539: INFO: Logging kubelet events for node i-02b09d19b2096261b Jan 9 06:49:06.684: INFO: Logging pods the kubelet thinks is on node i-02b09d19b2096261b Jan 9 06:49:06.839: INFO: etcd-manager-main-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:06.839: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:49:06.839: INFO: kube-apiserver-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+2 container statuses recorded) Jan 9 06:49:06.839: INFO: Container healthcheck ready: true, restart count 0 Jan 9 06:49:06.839: INFO: Container kube-apiserver ready: true, restart count 1 Jan 9 06:49:06.839: INFO: cilium-97g4h started at 2023-01-09 06:24:28 +0000 UTC (1+1 container statuses recorded) Jan 9 06:49:06.839: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:49:06.839: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:49:06.839: INFO: dns-controller-69987775c6-mv7h2 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:06.839: INFO: Container dns-controller ready: true, restart count 0 Jan 9 06:49:06.839: INFO: cilium-operator-5dd44dc49f-fgc7c started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:06.839: INFO: Container cilium-operator ready: true, restart count 0 Jan 9 06:49:06.839: INFO: kube-controller-manager-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:06.839: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 9 06:49:06.839: INFO: kube-scheduler-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:06.839: INFO: Container kube-scheduler ready: true, restart count 0 Jan 9 06:49:06.839: INFO: etcd-manager-events-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:06.839: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:49:06.839: INFO: aws-cloud-controller-manager-kj4b6 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:06.839: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 9 06:49:06.839: INFO: kops-controller-8qjsv started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:06.839: INFO: Container kops-controller ready: true, restart count 0 Jan 9 06:49:06.839: INFO: ebs-csi-node-kblmh started at 2023-01-09 06:24:28 +0000 UTC (0+3 container statuses recorded) Jan 9 06:49:06.839: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:49:06.839: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:49:06.839: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:49:06.839: INFO: ebs-csi-controller-89b649c94-zwk75 started at 2023-01-09 06:24:28 +0000 UTC (0+5 container statuses recorded) Jan 9 06:49:06.839: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:49:06.839: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:49:06.839: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:49:06.839: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:49:06.839: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:49:07.353: INFO: Latency metrics for node i-02b09d19b2096261b Jan 9 06:49:07.353: INFO: Logging node info for node i-0585809501bfc393d Jan 9 06:49:07.495: INFO: Node Info: &Node{ObjectMeta:{i-0585809501bfc393d 8c7c1c77-e673-43a0-81a4-fcf97493bcd8 37401 0 2023-01-09 06:25:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0585809501bfc393d kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-0585809501bfc393d topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.147 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0585809501bfc393d"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-09 06:48:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0585809501bfc393d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:48:44 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:48:44 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:48:44 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:48:44 +0000 UTC,LastTransitionTime:2023-01-09 06:25:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.147,},NodeAddress{Type:ExternalIP,Address:54.94.242.160,},NodeAddress{Type:InternalDNS,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-242-160.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec206fc38cf93b7ea12ba838ad958aa3,SystemUUID:ec206fc3-8cf9-3b7e-a12b-a838ad958aa3,BootID:afb66846-6976-49bf-9e61-b54e5af8a3eb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:49:07.495: INFO: Logging kubelet events for node i-0585809501bfc393d Jan 9 06:49:07.639: INFO: Logging pods the kubelet thinks is on node i-0585809501bfc393d Jan 9 06:49:07.788: INFO: test-webserver-1a3300d8-d9cf-488f-889d-601ebbfc198f started at 2023-01-09 06:47:43 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:07.788: INFO: Container test-webserver ready: true, restart count 0 Jan 9 06:49:07.788: INFO: test-rolling-update-with-lb-56d9b65d4c-dknvd started at 2023-01-09 06:46:45 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:07.788: INFO: Container agnhost ready: true, restart count 0 Jan 9 06:49:07.788: INFO: pod-service-account-defaultsa-mountspec started at 2023-01-09 06:48:50 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:07.788: INFO: Container token-test ready: false, restart count 0 Jan 9 06:49:07.788: INFO: fail-once-non-local-ngx6r started at 2023-01-09 06:48:52 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:07.788: INFO: Container c ready: false, restart count 0 Jan 9 06:49:07.788: INFO: fail-once-non-local-wc27l started at 2023-01-09 06:48:52 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:07.788: INFO: Container c ready: false, restart count 0 Jan 9 06:49:07.788: INFO: ss2-0 started at 2023-01-09 06:48:25 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:07.788: INFO: Container webserver ready: true, restart count 0 Jan 9 06:49:07.788: INFO: sample-crd-conversion-webhook-deployment-74ff66dd47-6h4cj started at 2023-01-09 06:48:57 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:07.788: INFO: Container sample-crd-conversion-webhook ready: true, restart count 0 Jan 9 06:49:07.788: INFO: cilium-f5j8w started at 2023-01-09 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 9 06:49:07.788: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:49:07.788: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:49:07.788: INFO: hostexec-i-0585809501bfc393d-wfwbq started at 2023-01-09 06:48:47 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:07.788: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:49:07.788: INFO: rc-test-rdr74 started at 2023-01-09 06:48:48 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:07.788: INFO: Container rc-test ready: true, restart count 0 Jan 9 06:49:07.788: INFO: externalname-service-jrp7x started at 2023-01-09 06:48:52 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:07.788: INFO: Container externalname-service ready: true, restart count 0 Jan 9 06:49:07.788: INFO: pod-service-account-nomountsa-mountspec started at 2023-01-09 06:48:50 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:07.788: INFO: Container token-test ready: false, restart count 0 Jan 9 06:49:07.788: INFO: ebs-csi-node-6djgn started at 2023-01-09 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 9 06:49:07.788: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:49:07.788: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:49:07.788: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:49:07.788: INFO: hostexec-i-0585809501bfc393d-7m5v6 started at 2023-01-09 06:48:24 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:07.788: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:49:07.788: INFO: pod-subpath-test-preprovisionedpv-pdmw started at <nil> (0+0 container statuses recorded) Jan 9 06:49:07.788: INFO: local-client started at <nil> (0+0 container statuses recorded) Jan 9 06:49:07.788: INFO: service-deletion-b9lll started at <nil> (0+0 container statuses recorded) Jan 9 06:49:08.299: INFO: Latency metrics for node i-0585809501bfc393d Jan 9 06:49:08.299: INFO: Logging node info for node i-06d5855698aa7cb08 Jan 9 06:49:08.441: INFO: Node Info: &Node{ObjectMeta:{i-06d5855698aa7cb08 bdfd8f9d-5aa1-4cff-806a-849d1c29fcb6 38270 0 2023-01-09 06:25:34 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06d5855698aa7cb08 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-06d5855698aa7cb08 topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.46.163 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-9781":"i-06d5855698aa7cb08","ebs.csi.aws.com":"i-06d5855698aa7cb08"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2023-01-09 06:49:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06d5855698aa7cb08,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:48:44 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:48:44 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:48:44 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:48:44 +0000 UTC,LastTransitionTime:2023-01-09 06:25:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.46.163,},NodeAddress{Type:ExternalIP,Address:18.228.235.107,},NodeAddress{Type:InternalDNS,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-107.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29e10a0890d67138c4eff5b197ee61,SystemUUID:ec29e10a-0890-d671-38c4-eff5b197ee61,BootID:9352599c-af91-4c75-860f-bb9a2789a998,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:49:08.442: INFO: Logging kubelet events for node i-06d5855698aa7cb08 Jan 9 06:49:08.587: INFO: Logging pods the kubelet thinks is on node i-06d5855698aa7cb08 Jan 9 06:49:08.733: INFO: pod-cf3c28a4-b4a9-4c1a-ae5c-fcebf74aa705 started at <nil> (0+0 container statuses recorded) Jan 9 06:49:08.733: INFO: test-ss-0 started at <nil> (0+0 container statuses recorded) Jan 9 06:49:08.733: INFO: csi-mockplugin-0 started at 2023-01-09 06:48:16 +0000 UTC (0+3 container statuses recorded) Jan 9 06:49:08.733: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:49:08.733: INFO: Container driver-registrar ready: true, restart count 0 Jan 9 06:49:08.733: INFO: Container mock ready: true, restart count 0 Jan 9 06:49:08.733: INFO: execpodvp9rp started at 2023-01-09 06:49:01 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:08.733: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:49:08.733: INFO: ebs-csi-node-6dzh8 started at 2023-01-09 06:25:35 +0000 UTC (0+3 container statuses recorded) Jan 9 06:49:08.733: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:49:08.733: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:49:08.733: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:49:08.733: INFO: hairpin started at 2023-01-09 06:49:06 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:08.733: INFO: Container agnhost-container ready: false, restart count 0 Jan 9 06:49:08.733: INFO: coredns-559769c974-qdhtp started at 2023-01-09 06:26:21 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:08.733: INFO: Container coredns ready: true, restart count 0 Jan 9 06:49:08.733: INFO: cilium-wbn8d started at 2023-01-09 06:25:35 +0000 UTC (1+1 container statuses recorded) Jan 9 06:49:08.733: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:49:08.733: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:49:08.733: INFO: pod-3398bd8e-a13f-4cce-aa72-060e81c3979b started at 2023-01-09 06:49:04 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:08.733: INFO: Container test-container ready: false, restart count 0 Jan 9 06:49:09.246: INFO: Latency metrics for node i-06d5855698aa7cb08 Jan 9 06:49:09.246: INFO: Logging node info for node i-087c2326a312024ef Jan 9 06:49:09.388: INFO: Node Info: &Node{ObjectMeta:{i-087c2326a312024ef 45db8cc8-250b-411a-b344-d6bbffe46b7d 37887 0 2023-01-09 06:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-087c2326a312024ef kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-087c2326a312024ef topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.218 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-5935":"i-087c2326a312024ef","ebs.csi.aws.com":"i-087c2326a312024ef"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:48:42 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-09 06:48:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-087c2326a312024ef,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:48:42 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:48:42 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:48:42 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:48:42 +0000 UTC,LastTransitionTime:2023-01-09 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.218,},NodeAddress{Type:ExternalIP,Address:18.230.17.62,},NodeAddress{Type:InternalDNS,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-17-62.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec28fbdf9f29233021716214b9f8da3a,SystemUUID:ec28fbdf-9f29-2330-2171-6214b9f8da3a,BootID:4dbf8c75-8514-419f-b443-1a44f89f420f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-5935^8bb39a63-8fe9-11ed-bb2f-b26f32bd1839 kubernetes.io/csi/csi-hostpath-ephemeral-5935^9db4cadf-8fe9-11ed-bb2f-b26f32bd1839 kubernetes.io/csi/ebs.csi.aws.com^vol-0d49342c84d2a787d],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-5935^9db4cadf-8fe9-11ed-bb2f-b26f32bd1839,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-5935^8bb39a63-8fe9-11ed-bb2f-b26f32bd1839,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d49342c84d2a787d,DevicePath:,},},Config:nil,},} Jan 9 06:49:09.388: INFO: Logging kubelet events for node i-087c2326a312024ef Jan 9 06:49:09.537: INFO: Logging pods the kubelet thinks is on node i-087c2326a312024ef Jan 9 06:49:09.686: INFO: externalname-service-rdxjw started at 2023-01-09 06:48:52 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:09.686: INFO: Container externalname-service ready: true, restart count 0 Jan 9 06:49:09.686: INFO: pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d started at 2023-01-09 06:39:05 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:09.686: INFO: Container write-pod ready: true, restart count 0 Jan 9 06:49:09.686: INFO: ss2-1 started at 2023-01-09 06:48:17 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:09.686: INFO: Container webserver ready: true, restart count 0 Jan 9 06:49:09.686: INFO: agnhost-primary-fjszc started at 2023-01-09 06:48:30 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:09.686: INFO: Container agnhost-primary ready: false, restart count 0 Jan 9 06:49:09.686: INFO: pod-secrets-0869eedc-a3dd-4686-bb06-1c943e04de65 started at 2023-01-09 06:48:32 +0000 UTC (0+3 container statuses recorded) Jan 9 06:49:09.686: INFO: Container creates-volume-test ready: true, restart count 0 Jan 9 06:49:09.686: INFO: Container dels-volume-test ready: true, restart count 0 Jan 9 06:49:09.686: INFO: Container upds-volume-test ready: true, restart count 0 Jan 9 06:49:09.686: INFO: ebs-csi-node-jvd2b started at 2023-01-09 06:25:35 +0000 UTC (0+3 container statuses recorded) Jan 9 06:49:09.686: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:49:09.686: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:49:09.686: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:49:09.686: INFO: cilium-gw6mm started at 2023-01-09 06:25:35 +0000 UTC (1+1 container statuses recorded) Jan 9 06:49:09.686: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:49:09.686: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:49:09.686: INFO: test-rolling-update-with-lb-56d9b65d4c-489qj started at 2023-01-09 06:46:45 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:09.686: INFO: Container agnhost ready: true, restart count 0 Jan 9 06:49:09.686: INFO: termination-message-container0ee3c08c-528e-4b01-80be-f4ba9d4fc452 started at 2023-01-09 06:49:05 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:09.686: INFO: Container termination-message-container ready: false, restart count 0 Jan 9 06:49:09.686: INFO: pfpod started at 2023-01-09 06:49:02 +0000 UTC (0+2 container statuses recorded) Jan 9 06:49:09.686: INFO: Container portforwardtester ready: true, restart count 0 Jan 9 06:49:09.686: INFO: Container readiness ready: true, restart count 0 Jan 9 06:49:09.686: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:47:08 +0000 UTC (0+7 container statuses recorded) Jan 9 06:49:09.686: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:49:09.686: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:49:09.686: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:49:09.686: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 9 06:49:09.686: INFO: Container hostpath ready: true, restart count 0 Jan 9 06:49:09.686: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:49:09.686: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:49:09.686: INFO: inline-volume-tester-n6fxp started at 2023-01-09 06:47:55 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:09.686: INFO: Container csi-volume-tester ready: true, restart count 0 Jan 9 06:49:09.686: INFO: inline-volume-tester2-xgtbh started at 2023-01-09 06:48:26 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:09.686: INFO: Container csi-volume-tester ready: true, restart count 0 Jan 9 06:49:10.427: INFO: Latency metrics for node i-087c2326a312024ef Jan 9 06:49:10.427: INFO: Logging node info for node i-0c0b18eb32e7da83d Jan 9 06:49:10.569: INFO: Node Info: &Node{ObjectMeta:{i-0c0b18eb32e7da83d f1550edf-43cd-451c-bf01-eb92d6980d45 38306 0 2023-01-09 06:25:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0c0b18eb32e7da83d kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-0c0b18eb32e7da83d topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.55.27 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-9394":"i-0c0b18eb32e7da83d","ebs.csi.aws.com":"i-0c0b18eb32e7da83d"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2023-01-09 06:49:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-09 06:49:05 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c0b18eb32e7da83d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:49:02 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:49:02 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:49:02 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:49:02 +0000 UTC,LastTransitionTime:2023-01-09 06:25:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.55.27,},NodeAddress{Type:ExternalIP,Address:18.228.15.186,},NodeAddress{Type:InternalDNS,Address:i-0c0b18eb32e7da83d.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0c0b18eb32e7da83d.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-15-186.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec267e00a4086801bd60f1191c81f587,SystemUUID:ec267e00-a408-6801-bd60-f1191c81f587,BootID:7c624b34-f575-4c40-8226-aef6612a1a56,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-02aed7c85535fd1be kubernetes.io/csi/ebs.csi.aws.com^vol-0a7ca5f26c8b1cb76],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-02aed7c85535fd1be,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0a7ca5f26c8b1cb76,DevicePath:,},},Config:nil,},} Jan 9 06:49:10.569: INFO: Logging kubelet events for node i-0c0b18eb32e7da83d Jan 9 06:49:10.715: INFO: Logging pods the kubelet thinks is on node i-0c0b18eb32e7da83d Jan 9 06:49:10.868: INFO: ss2-2 started at 2023-01-09 06:48:52 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:10.868: INFO: Container webserver ready: true, restart count 0 Jan 9 06:49:10.868: INFO: inline-volume-tester-vx5kj started at 2023-01-09 06:48:29 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:10.868: INFO: Container csi-volume-tester ready: true, restart count 0 Jan 9 06:49:10.868: INFO: agnhost-primary-t29kh started at 2023-01-09 06:48:29 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:10.868: INFO: Container agnhost-primary ready: false, restart count 0 Jan 9 06:49:10.868: INFO: cilium-vx7vq started at 2023-01-09 06:25:34 +0000 UTC (1+1 container statuses recorded) Jan 9 06:49:10.868: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:49:10.868: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:49:10.868: INFO: coredns-559769c974-md8gz started at 2023-01-09 06:25:51 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:10.868: INFO: Container coredns ready: true, restart count 0 Jan 9 06:49:10.868: INFO: coredns-autoscaler-7cb5c5b969-4vwwx started at 2023-01-09 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:10.868: INFO: Container autoscaler ready: true, restart count 0 Jan 9 06:49:10.868: INFO: busybox-565ea581-e79e-4d32-a8c9-4d3cc08e7b01 started at 2023-01-09 06:46:32 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:10.868: INFO: Container busybox ready: true, restart count 0 Jan 9 06:49:10.868: INFO: rs-sf75v started at 2023-01-09 06:48:27 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:10.868: INFO: Container donothing ready: false, restart count 0 Jan 9 06:49:10.868: INFO: ebs-csi-node-m5kg7 started at 2023-01-09 06:25:34 +0000 UTC (0+3 container statuses recorded) Jan 9 06:49:10.868: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:49:10.868: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:49:10.868: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:49:10.868: INFO: test-rolling-update-with-lb-56d9b65d4c-7rvk6 started at 2023-01-09 06:46:45 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:10.868: INFO: Container agnhost ready: true, restart count 0 Jan 9 06:49:10.868: INFO: pod-service-account-mountsa-mountspec started at 2023-01-09 06:48:50 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:10.868: INFO: Container token-test ready: false, restart count 0 Jan 9 06:49:10.868: INFO: inline-volume-tester-rzfwn started at 2023-01-09 06:48:54 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:10.868: INFO: Container csi-volume-tester ready: true, restart count 0 Jan 9 06:49:10.868: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:48:53 +0000 UTC (0+7 container statuses recorded) Jan 9 06:49:10.868: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:49:10.868: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:49:10.868: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:49:10.868: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 9 06:49:10.868: INFO: Container hostpath ready: true, restart count 0 Jan 9 06:49:10.868: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:49:10.868: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:49:10.868: INFO: startup-script started at 2023-01-09 06:47:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:49:10.868: INFO: Container startup-script ready: true, restart count 0 Jan 9 06:49:11.425: INFO: Latency metrics for node i-0c0b18eb32e7da83d [DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "fsgroupchangepolicy-5880" for this suite. �[38;5;243m01/09/23 06:49:11.425�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-apps\]\sDeployment\sdeployment\sshould\ssupport\srollover\s\[Conformance\]$'
test/e2e/apps/deployment.go:904 k8s.io/kubernetes/test/e2e/apps.testRolloverDeployment(0xc0009da1d8?) test/e2e/apps/deployment.go:904 +0x4d1 k8s.io/kubernetes/test/e2e/apps.glob..func5.7() test/e2e/apps/deployment.go:133 +0x1dfrom junit_01.xml
[BeforeEach] [sig-apps] Deployment set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/09/23 06:39:31.672�[0m Jan 9 06:39:31.672: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename deployment �[38;5;243m01/09/23 06:39:31.673�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/09/23 06:39:32.099�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/09/23 06:39:32.382�[0m [BeforeEach] [sig-apps] Deployment test/e2e/framework/metrics/init/init.go:31 [BeforeEach] [sig-apps] Deployment test/e2e/apps/deployment.go:91 [It] deployment should support rollover [Conformance] test/e2e/apps/deployment.go:132 Jan 9 06:39:32.951: INFO: Pod name rollover-pod: Found 1 pods out of 1 �[1mSTEP:�[0m ensuring each pod is running �[38;5;243m01/09/23 06:39:32.951�[0m Jan 9 06:39:32.952: INFO: Waiting up to 5m0s for pod "test-rollover-controller-75rk6" in namespace "deployment-7519" to be "running" Jan 9 06:39:33.093: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 141.700797ms Jan 9 06:39:35.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2.284656076s Jan 9 06:39:37.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4.285197439s Jan 9 06:39:39.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 6.28482201s Jan 9 06:39:41.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 8.284654605s Jan 9 06:39:43.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 10.284027532s Jan 9 06:39:45.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 12.284292957s Jan 9 06:39:47.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 14.284503044s Jan 9 06:39:49.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 16.285471961s Jan 9 06:39:51.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 18.284077177s Jan 9 06:39:53.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 20.284615096s Jan 9 06:39:55.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 22.284988847s Jan 9 06:39:57.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 24.283967144s Jan 9 06:39:59.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 26.284078617s Jan 9 06:40:01.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 28.284955333s Jan 9 06:40:03.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 30.284415247s Jan 9 06:40:05.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 32.283975915s Jan 9 06:40:07.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 34.284024306s Jan 9 06:40:09.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 36.284595739s Jan 9 06:40:11.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 38.283977571s Jan 9 06:40:13.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 40.284466245s Jan 9 06:40:15.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 42.283864265s Jan 9 06:40:17.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 44.284316913s Jan 9 06:40:19.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 46.284631297s Jan 9 06:40:21.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 48.284608389s Jan 9 06:40:23.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 50.284247783s Jan 9 06:40:25.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 52.28482507s Jan 9 06:40:27.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 54.28391111s Jan 9 06:40:29.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 56.284719754s Jan 9 06:40:31.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 58.284397848s Jan 9 06:40:33.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.283665133s Jan 9 06:40:35.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.284745949s Jan 9 06:40:37.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.284598683s Jan 9 06:40:39.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.284542861s Jan 9 06:40:41.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.284840538s Jan 9 06:40:43.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.284186567s Jan 9 06:40:45.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.285221583s Jan 9 06:40:47.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.284327258s Jan 9 06:40:49.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.283932927s Jan 9 06:40:51.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.283892309s Jan 9 06:40:53.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.28455074s Jan 9 06:40:55.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.284533996s Jan 9 06:40:57.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.283814162s Jan 9 06:40:59.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.284973514s Jan 9 06:41:01.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.284407671s Jan 9 06:41:03.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.285662106s Jan 9 06:41:05.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.284334413s Jan 9 06:41:07.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.283957731s Jan 9 06:41:09.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.284389688s Jan 9 06:41:11.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.285058499s Jan 9 06:41:13.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.284450059s Jan 9 06:41:15.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.285288861s Jan 9 06:41:17.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.28496304s Jan 9 06:41:19.241: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.289402681s Jan 9 06:41:21.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.284470803s Jan 9 06:41:23.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.284304424s Jan 9 06:41:25.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.284622632s Jan 9 06:41:27.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.284502738s Jan 9 06:41:29.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.283828553s Jan 9 06:41:31.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.284872779s Jan 9 06:41:33.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.285059312s Jan 9 06:41:35.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.283998332s Jan 9 06:41:37.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.284501062s Jan 9 06:41:39.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.283965868s Jan 9 06:41:41.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.28399724s Jan 9 06:41:43.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.284096078s Jan 9 06:41:45.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.283642511s Jan 9 06:41:47.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.284636358s Jan 9 06:41:49.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.284453341s Jan 9 06:41:51.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.283873279s Jan 9 06:41:53.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.284561138s Jan 9 06:41:55.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.2848374s Jan 9 06:41:57.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.284170953s Jan 9 06:41:59.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.284507911s Jan 9 06:42:01.238: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.285865045s Jan 9 06:42:03.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.283907421s Jan 9 06:42:05.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.284385116s Jan 9 06:42:07.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.284961829s Jan 9 06:42:09.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.283603521s Jan 9 06:42:11.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.284093926s Jan 9 06:42:13.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.284654371s Jan 9 06:42:15.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.283881088s Jan 9 06:42:17.259: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.307275854s Jan 9 06:42:19.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.284846662s Jan 9 06:42:21.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.283890851s Jan 9 06:42:23.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.28419464s Jan 9 06:42:25.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.284785929s Jan 9 06:42:27.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.283697313s Jan 9 06:42:29.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.283829305s Jan 9 06:42:31.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.284912197s Jan 9 06:42:33.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.284632619s Jan 9 06:42:35.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.2836936s Jan 9 06:42:37.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.28398797s Jan 9 06:42:39.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.284052695s Jan 9 06:42:41.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.284784867s Jan 9 06:42:43.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.283760121s Jan 9 06:42:45.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.283629303s Jan 9 06:42:47.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.284909734s Jan 9 06:42:49.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.285668088s Jan 9 06:42:51.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.284129347s Jan 9 06:42:53.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.284443964s Jan 9 06:42:55.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.284747808s Jan 9 06:42:57.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.284403769s Jan 9 06:42:59.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.283994065s Jan 9 06:43:01.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.284887696s Jan 9 06:43:03.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.284712658s Jan 9 06:43:05.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.284948614s Jan 9 06:43:07.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.284824156s Jan 9 06:43:09.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.283990613s Jan 9 06:43:11.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.284770843s Jan 9 06:43:13.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.284821585s Jan 9 06:43:15.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.283963585s Jan 9 06:43:17.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.283832878s Jan 9 06:43:19.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.284850591s Jan 9 06:43:21.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.28456591s Jan 9 06:43:23.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.284749562s Jan 9 06:43:25.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.284918847s Jan 9 06:43:27.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.284533637s Jan 9 06:43:29.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.283604635s Jan 9 06:43:31.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.284604415s Jan 9 06:43:33.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.284338665s Jan 9 06:43:35.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.283971089s Jan 9 06:43:37.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.284470135s Jan 9 06:43:39.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.283662843s Jan 9 06:43:41.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.284556238s Jan 9 06:43:43.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.283947824s Jan 9 06:43:45.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.283995803s Jan 9 06:43:47.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.284971342s Jan 9 06:43:49.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.284172277s Jan 9 06:43:51.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.285013252s Jan 9 06:43:53.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.28375237s Jan 9 06:43:55.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.284620494s Jan 9 06:43:57.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.284789585s Jan 9 06:43:59.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.284249001s Jan 9 06:44:01.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.284459322s Jan 9 06:44:03.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.285330789s Jan 9 06:44:05.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.284636976s Jan 9 06:44:07.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.284379244s Jan 9 06:44:09.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.283740372s Jan 9 06:44:11.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.28461471s Jan 9 06:44:13.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.284079994s Jan 9 06:44:15.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.283749537s Jan 9 06:44:17.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.284448801s Jan 9 06:44:19.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.284752305s Jan 9 06:44:21.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.284366392s Jan 9 06:44:23.235: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.283758685s Jan 9 06:44:25.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.284467567s Jan 9 06:44:27.237: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.284982417s Jan 9 06:44:29.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.284240198s Jan 9 06:44:31.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.284236637s Jan 9 06:44:33.236: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.284498802s Jan 9 06:44:33.378: INFO: Pod "test-rollover-controller-75rk6": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.426112875s Jan 9 06:44:33.378: INFO: Unexpected error: error in waiting for pods to come up: failed to wait for pods running: [timed out while waiting for pod deployment-7519/test-rollover-controller-75rk6 to be running]: <*errors.errorString | 0xc00132e670>: { s: "failed to wait for pods running: [timed out while waiting for pod deployment-7519/test-rollover-controller-75rk6 to be running]", } Jan 9 06:44:33.378: FAIL: error in waiting for pods to come up: failed to wait for pods running: [timed out while waiting for pod deployment-7519/test-rollover-controller-75rk6 to be running]: failed to wait for pods running: [timed out while waiting for pod deployment-7519/test-rollover-controller-75rk6 to be running] Full Stack Trace k8s.io/kubernetes/test/e2e/apps.testRolloverDeployment(0xc0009da1d8?) test/e2e/apps/deployment.go:904 +0x4d1 k8s.io/kubernetes/test/e2e/apps.glob..func5.7() test/e2e/apps/deployment.go:133 +0x1d [AfterEach] [sig-apps] Deployment test/e2e/apps/deployment.go:84 Jan 9 06:44:33.521: INFO: Log out all the ReplicaSets if there is no deployment created Jan 9 06:44:33.664: INFO: ReplicaSet "test-rollover-controller": &ReplicaSet{ObjectMeta:{test-rollover-controller deployment-7519 edcde0cd-8528-4404-8152-46834ddcbbcc 28400 1 2023-01-09 06:39:32 +0000 UTC <nil> <nil> map[name:rollover-pod pod:httpd] map[] [] [] [{e2e.test Update apps/v1 2023-01-09 06:39:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{".":{},"f:name":{},"f:pod":{}}},"f:spec":{"f:replicas":{},"f:selector":{},"f:template":{"f:metadata":{"f:labels":{".":{},"f:name":{},"f:pod":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"httpd\"}":{".":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}}} } {kube-controller-manager Update apps/v1 2023-01-09 06:39:32 +0000 UTC FieldsV1 {"f:status":{"f:fullyLabeledReplicas":{},"f:observedGeneration":{},"f:replicas":{}}} status}]},Spec:ReplicaSetSpec{Replicas:*1,Selector:&v1.LabelSelector{MatchLabels:map[string]string{name: rollover-pod,pod: httpd,},MatchExpressions:[]LabelSelectorRequirement{},},Template:{{ 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[name:rollover-pod pod:httpd] map[] [] [] []} {[] [] [{httpd registry.k8s.io/e2e-test-images/httpd:2.4.38-4 [] [] [] [] [] {map[] map[] []} [] [] nil nil nil nil /dev/termination-log File IfNotPresent nil false false false}] [] Always 0xc00435ef68 <nil> ClusterFirst map[] <nil> false false false <nil> PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,} [] nil default-scheduler [] [] <nil> nil [] <nil> <nil> <nil> map[] [] <nil> nil <nil> [] []}},MinReadySeconds:0,},Status:ReplicaSetStatus{Replicas:1,FullyLabeledReplicas:1,ObservedGeneration:1,ReadyReplicas:0,AvailableReplicas:0,Conditions:[]ReplicaSetCondition{},},} Jan 9 06:44:33.807: INFO: pod: "test-rollover-controller-75rk6": &Pod{ObjectMeta:{test-rollover-controller-75rk6 test-rollover-controller- deployment-7519 877ba6e1-d747-464e-8f67-1a70111a6959 28434 0 2023-01-09 06:39:32 +0000 UTC <nil> <nil> map[name:rollover-pod pod:httpd] map[] [{apps/v1 ReplicaSet test-rollover-controller edcde0cd-8528-4404-8152-46834ddcbbcc 0xc00435f247 0xc00435f248}] [] [{kube-controller-manager Update v1 2023-01-09 06:39:32 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{},"f:labels":{".":{},"f:name":{},"f:pod":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"edcde0cd-8528-4404-8152-46834ddcbbcc\"}":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"httpd\"}":{".":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}} } {kubelet Update v1 2023-01-09 06:39:33 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:startTime":{}}} status}]},Spec:PodSpec{Volumes:[]Volume{Volume{Name:kube-api-access-w8rqg,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:nil,NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:&ProjectedVolumeSource{Sources:[]VolumeProjection{VolumeProjection{Secret:nil,DownwardAPI:nil,ConfigMap:nil,ServiceAccountToken:&ServiceAccountTokenProjection{Audience:,ExpirationSeconds:*3607,Path:token,},},VolumeProjection{Secret:nil,DownwardAPI:nil,ConfigMap:&ConfigMapProjection{LocalObjectReference:LocalObjectReference{Name:kube-root-ca.crt,},Items:[]KeyToPath{KeyToPath{Key:ca.crt,Path:ca.crt,Mode:nil,},},Optional:nil,},ServiceAccountToken:nil,},VolumeProjection{Secret:nil,DownwardAPI:&DownwardAPIProjection{Items:[]DownwardAPIVolumeFile{DownwardAPIVolumeFile{Path:namespace,FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,Mode:nil,},},},ConfigMap:nil,ServiceAccountToken:nil,},},DefaultMode:*420,},StorageOS:nil,CSI:nil,Ephemeral:nil,},},},Containers:[]Container{Container{Name:httpd,Image:registry.k8s.io/e2e-test-images/httpd:2.4.38-4,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-w8rqg,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,},},RestartPolicy:Always,TerminationGracePeriodSeconds:*0,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:i-0585809501bfc393d,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,},ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{Toleration{Key:node.kubernetes.io/not-ready,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},Toleration{Key:node.kubernetes.io/unreachable,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},},HostAliases:[]HostAlias{},PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:*true,PreemptionPolicy:*PreemptLowerPriority,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},SetHostnameAsFQDN:nil,OS:nil,HostUsers:nil,SchedulingGates:[]PodSchedulingGate{},ResourceClaims:[]PodResourceClaim{},},Status:PodStatus{Phase:Pending,Conditions:[]PodCondition{PodCondition{Type:Initialized,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2023-01-09 06:39:32 +0000 UTC,Reason:,Message:,},PodCondition{Type:Ready,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2023-01-09 06:39:32 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:ContainersReady,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2023-01-09 06:39:32 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [httpd],},PodCondition{Type:PodScheduled,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2023-01-09 06:39:32 +0000 UTC,Reason:,Message:,},},Message:,Reason:,HostIP:172.20.40.147,PodIP:,StartTime:2023-01-09 06:39:32 +0000 UTC,ContainerStatuses:[]ContainerStatus{ContainerStatus{Name:httpd,State:ContainerState{Waiting:&ContainerStateWaiting{Reason:ContainerCreating,Message:,},Running:nil,Terminated:nil,},LastTerminationState:ContainerState{Waiting:nil,Running:nil,Terminated:nil,},Ready:false,RestartCount:0,Image:registry.k8s.io/e2e-test-images/httpd:2.4.38-4,ImageID:,ContainerID:,Started:*false,},},QOSClass:BestEffort,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{},EphemeralContainerStatuses:[]ContainerStatus{},},} [AfterEach] [sig-apps] Deployment test/e2e/framework/node/init/init.go:32 Jan 9 06:44:33.807: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-apps] Deployment test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-apps] Deployment dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/09/23 06:44:33.95�[0m �[1mSTEP:�[0m Collecting events from namespace "deployment-7519". �[38;5;243m01/09/23 06:44:33.95�[0m �[1mSTEP:�[0m Found 12 events. �[38;5;243m01/09/23 06:44:34.092�[0m Jan 9 06:44:34.092: INFO: At 2023-01-09 06:39:32 +0000 UTC - event for test-rollover-controller: {replicaset-controller } SuccessfulCreate: Created pod: test-rollover-controller-75rk6 Jan 9 06:44:34.093: INFO: At 2023-01-09 06:39:32 +0000 UTC - event for test-rollover-controller-75rk6: {default-scheduler } Scheduled: Successfully assigned deployment-7519/test-rollover-controller-75rk6 to i-0585809501bfc393d Jan 9 06:44:34.093: INFO: At 2023-01-09 06:39:33 +0000 UTC - event for test-rollover-controller-75rk6: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "65681580b6b5947f8240122a115ce24a9ba7fd604fe3f0c8df446c04aa83b132": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:34.093: INFO: At 2023-01-09 06:39:47 +0000 UTC - event for test-rollover-controller-75rk6: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bb4490bd418c76b5715ae581314f3fc4b717de2a28a27299607416ee6948fc6a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:34.093: INFO: At 2023-01-09 06:40:00 +0000 UTC - event for test-rollover-controller-75rk6: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "acc4fb3277cb2e5ba152839a022f3ff2bdbf465d5f842e337e2e91b959df4d9b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:34.093: INFO: At 2023-01-09 06:40:13 +0000 UTC - event for test-rollover-controller-75rk6: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "37d14373a4a38fbfc5f5bfff9ecdeb3ac816c925e8a3e8d7c176dbe69740a180": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:34.093: INFO: At 2023-01-09 06:40:29 +0000 UTC - event for test-rollover-controller-75rk6: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6b2a077fd3a2e92d818f52600f1e00d589d5a1f59bde611272154eef0dcc6a2b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:34.093: INFO: At 2023-01-09 06:40:42 +0000 UTC - event for test-rollover-controller-75rk6: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5be7e732f70ffa222a0155bb798b596f6faa7ab373769e62b10368d7fcadffc6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:34.093: INFO: At 2023-01-09 06:40:57 +0000 UTC - event for test-rollover-controller-75rk6: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "07a76b448116a2b987786e221266bf511890ec689852bbdc89fb02716fb9289f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:34.093: INFO: At 2023-01-09 06:41:08 +0000 UTC - event for test-rollover-controller-75rk6: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "fc8ba7e5c26fed9d49804972d91d214d93e631aa57df73c6512f2a15dea4aca8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:34.093: INFO: At 2023-01-09 06:41:23 +0000 UTC - event for test-rollover-controller-75rk6: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6b94396f84eb5b02f96c0a84438912a1ecb066fed93e367de198e44886cb28e4": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:34.093: INFO: At 2023-01-09 06:41:38 +0000 UTC - event for test-rollover-controller-75rk6: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "44f1a9744b65229d71b0ed49e32891b467c3a8d10e9504b529d733f5d9a5b17a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:34.235: INFO: POD NODE PHASE GRACE CONDITIONS Jan 9 06:44:34.235: INFO: test-rollover-controller-75rk6 i-0585809501bfc393d Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:32 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:32 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:32 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:32 +0000 UTC }] Jan 9 06:44:34.235: INFO: Jan 9 06:44:34.378: INFO: Unable to fetch deployment-7519/test-rollover-controller-75rk6/httpd logs: the server rejected our request for an unknown reason (get pods test-rollover-controller-75rk6) Jan 9 06:44:34.522: INFO: Logging node info for node i-02b09d19b2096261b Jan 9 06:44:34.664: INFO: Node Info: &Node{ObjectMeta:{i-02b09d19b2096261b efa11f02-6d54-4902-bbeb-efecc1f02cd1 29640 0 2023-01-09 06:23:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-02b09d19b2096261b kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.12 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02b09d19b2096261b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-09 06:23:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-09 06:24:22 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-09 06:24:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-09 06:41:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-02b09d19b2096261b,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:24:43 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.12,},NodeAddress{Type:ExternalIP,Address:54.94.119.205,},NodeAddress{Type:InternalDNS,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-119-205.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2da51bd0e40b36975082ac186a369f,SystemUUID:ec2da51b-d0e4-0b36-9750-82ac186a369f,BootID:83891f80-d7aa-4da9-ab10-44706ab3b631,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43453930,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42803458,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967347,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:34.664: INFO: Logging kubelet events for node i-02b09d19b2096261b Jan 9 06:44:34.810: INFO: Logging pods the kubelet thinks is on node i-02b09d19b2096261b Jan 9 06:44:34.958: INFO: etcd-manager-main-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.958: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:44:34.958: INFO: kube-apiserver-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+2 container statuses recorded) Jan 9 06:44:34.958: INFO: Container healthcheck ready: true, restart count 0 Jan 9 06:44:34.958: INFO: Container kube-apiserver ready: true, restart count 1 Jan 9 06:44:34.958: INFO: cilium-97g4h started at 2023-01-09 06:24:28 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:34.958: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:34.958: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:34.958: INFO: dns-controller-69987775c6-mv7h2 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.958: INFO: Container dns-controller ready: true, restart count 0 Jan 9 06:44:34.958: INFO: cilium-operator-5dd44dc49f-fgc7c started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.958: INFO: Container cilium-operator ready: true, restart count 0 Jan 9 06:44:34.958: INFO: ebs-csi-node-kblmh started at 2023-01-09 06:24:28 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:34.958: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:34.958: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:34.958: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:34.958: INFO: ebs-csi-controller-89b649c94-zwk75 started at 2023-01-09 06:24:28 +0000 UTC (0+5 container statuses recorded) Jan 9 06:44:34.958: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:44:34.958: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:44:34.958: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:44:34.958: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:34.958: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:34.958: INFO: kube-controller-manager-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.958: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 9 06:44:34.958: INFO: kube-scheduler-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.958: INFO: Container kube-scheduler ready: true, restart count 0 Jan 9 06:44:34.958: INFO: etcd-manager-events-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.958: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:44:34.958: INFO: aws-cloud-controller-manager-kj4b6 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.958: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 9 06:44:34.958: INFO: kops-controller-8qjsv started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.958: INFO: Container kops-controller ready: true, restart count 0 Jan 9 06:44:35.459: INFO: Latency metrics for node i-02b09d19b2096261b Jan 9 06:44:35.459: INFO: Logging node info for node i-0585809501bfc393d Jan 9 06:44:35.601: INFO: Node Info: &Node{ObjectMeta:{i-0585809501bfc393d 8c7c1c77-e673-43a0-81a4-fcf97493bcd8 29271 0 2023-01-09 06:25:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0585809501bfc393d kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-0585809501bfc393d topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.147 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0585809501bfc393d"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-09 06:41:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0585809501bfc393d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.147,},NodeAddress{Type:ExternalIP,Address:54.94.242.160,},NodeAddress{Type:InternalDNS,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-242-160.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec206fc38cf93b7ea12ba838ad958aa3,SystemUUID:ec206fc3-8cf9-3b7e-a12b-a838ad958aa3,BootID:afb66846-6976-49bf-9e61-b54e5af8a3eb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:35.602: INFO: Logging kubelet events for node i-0585809501bfc393d Jan 9 06:44:35.748: INFO: Logging pods the kubelet thinks is on node i-0585809501bfc393d Jan 9 06:44:35.899: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-97f28 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.899: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-p6pfg started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.899: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-6hzsb started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.899: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:40:25 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:35.899: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:35.899: INFO: hostexec-i-0585809501bfc393d-v4gr6 started at 2023-01-09 06:39:24 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:35.899: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-cs2m5 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.899: INFO: test-rollover-controller-75rk6 started at 2023-01-09 06:39:32 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:35.899: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-bxrs7 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.899: INFO: netserver-0 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:35.899: INFO: cilium-f5j8w started at 2023-01-09 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:35.899: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:35.899: INFO: dns-test-bdcfcfaa-2367-4a35-bb69-c5dd9d18f66c started at 2023-01-09 06:44:29 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:35.899: INFO: Container jessie-querier ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container querier ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:35.899: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:39:17 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:35.899: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:35.899: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fs9d6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.899: INFO: alpine-nnp-true-7168054a-3362-4368-bc5e-54ecf20fc5ca started at 2023-01-09 06:39:32 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container alpine-nnp-true-7168054a-3362-4368-bc5e-54ecf20fc5ca ready: false, restart count 0 Jan 9 06:44:35.899: INFO: ebs-csi-node-6djgn started at 2023-01-09 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:35.899: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:35.899: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:35.899: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:35.899: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fm99c started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.899: INFO: execpodjqtvg started at 2023-01-09 06:42:07 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container agnhost-container ready: false, restart count 0 Jan 9 06:44:35.899: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-xtk85 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.899: INFO: alpine-nnp-false-4247b683-138d-4632-a6a3-6d844097c353 started at 2023-01-09 06:41:15 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container alpine-nnp-false-4247b683-138d-4632-a6a3-6d844097c353 ready: false, restart count 0 Jan 9 06:44:35.899: INFO: webserver-66b8c99f6-ls7f4 started at 2023-01-09 06:41:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:35.899: INFO: dns-test-a96d67de-eeb2-4e87-8d66-17c313a4ef5a started at 2023-01-09 06:40:06 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:35.899: INFO: Container jessie-querier ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container querier ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:35.899: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-qm6jm started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.899: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8rq57 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.899: INFO: pod-subpath-test-preprovisionedpv-wp87 started at 2023-01-09 06:39:43 +0000 UTC (2+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Init container init-volume-preprovisionedpv-wp87 ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Init container test-init-subpath-preprovisionedpv-wp87 ready: false, restart count 0 Jan 9 06:44:35.899: INFO: Container test-container-subpath-preprovisionedpv-wp87 ready: false, restart count 0 Jan 9 06:44:35.899: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-2jgj6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.899: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.550: INFO: Latency metrics for node i-0585809501bfc393d Jan 9 06:44:36.550: INFO: Logging node info for node i-06d5855698aa7cb08 Jan 9 06:44:36.692: INFO: Node Info: &Node{ObjectMeta:{i-06d5855698aa7cb08 bdfd8f9d-5aa1-4cff-806a-849d1c29fcb6 28608 0 2023-01-09 06:25:34 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06d5855698aa7cb08 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-06d5855698aa7cb08 topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.46.163 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-2682":"i-06d5855698aa7cb08","ebs.csi.aws.com":"i-06d5855698aa7cb08"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:39:47 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-09 06:39:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06d5855698aa7cb08,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.46.163,},NodeAddress{Type:ExternalIP,Address:18.228.235.107,},NodeAddress{Type:InternalDNS,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-107.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29e10a0890d67138c4eff5b197ee61,SystemUUID:ec29e10a-0890-d671-38c4-eff5b197ee61,BootID:9352599c-af91-4c75-860f-bb9a2789a998,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-2682^66d24229-8fe8-11ed-abbf-d2969ce11d4b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-2682^66d24229-8fe8-11ed-abbf-d2969ce11d4b,DevicePath:,},},Config:nil,},} Jan 9 06:44:36.692: INFO: Logging kubelet events for node i-06d5855698aa7cb08 Jan 9 06:44:36.836: INFO: Logging pods the kubelet thinks is on node i-06d5855698aa7cb08 Jan 9 06:44:36.997: INFO: webserver-7f5969cbc7-tz85c started at 2023-01-09 06:41:16 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.997: INFO: Container httpd ready: true, restart count 0 Jan 9 06:44:36.997: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-vwlp2 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.997: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.997: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pnbg6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.997: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.997: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-mkn6f started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.997: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.997: INFO: netserver-1 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.997: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:36.997: INFO: ebs-csi-node-6dzh8 started at 2023-01-09 06:25:35 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:36.997: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:36.997: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:36.997: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:36.997: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:42:19 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:36.997: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:36.997: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:36.997: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:36.997: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:36.997: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:36.997: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:36.997: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:36.997: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:39:00 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:36.997: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:44:36.997: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:44:36.997: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:44:36.997: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 9 06:44:36.997: INFO: Container hostpath ready: true, restart count 0 Jan 9 06:44:36.997: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:36.997: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:36.997: INFO: csi-mockplugin-0 started at 2023-01-09 06:40:46 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:36.997: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:36.997: INFO: Container driver-registrar ready: false, restart count 0 Jan 9 06:44:36.997: INFO: Container mock ready: false, restart count 0 Jan 9 06:44:36.997: INFO: coredns-559769c974-qdhtp started at 2023-01-09 06:26:21 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.997: INFO: Container coredns ready: true, restart count 0 Jan 9 06:44:36.997: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fncl4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.997: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.997: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8hxw6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.997: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.997: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-wg2gn started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.997: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.997: INFO: pod-d7eb61ad-d823-42ce-b057-d1397e0091be started at 2023-01-09 06:39:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.997: INFO: Container write-pod ready: false, restart count 0 Jan 9 06:44:36.997: INFO: cilium-wbn8d started at 2023-01-09 06:25:35 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:36.997: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:36.997: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:36.997: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-hpb4w started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.997: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.997: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kr2c4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.997: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.997: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-b42jp started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.997: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.997: INFO: hostexec-i-06d5855698aa7cb08-gdgwr started at 2023-01-09 06:39:24 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.997: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:37.630: INFO: Latency metrics for node i-06d5855698aa7cb08 Jan 9 06:44:37.630: INFO: Logging node info for node i-087c2326a312024ef Jan 9 06:44:37.772: INFO: Node Info: &Node{ObjectMeta:{i-087c2326a312024ef 45db8cc8-250b-411a-b344-d6bbffe46b7d 29314 0 2023-01-09 06:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-087c2326a312024ef kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-087c2326a312024ef topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.218 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-087c2326a312024ef"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:40:12 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-09 06:41:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-087c2326a312024ef,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.218,},NodeAddress{Type:ExternalIP,Address:18.230.17.62,},NodeAddress{Type:InternalDNS,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-17-62.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec28fbdf9f29233021716214b9f8da3a,SystemUUID:ec28fbdf-9f29-2330-2171-6214b9f8da3a,BootID:4dbf8c75-8514-419f-b443-1a44f89f420f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d49342c84d2a787d],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d49342c84d2a787d,DevicePath:,},},Config:nil,},} Jan 9 06:44:37.773: INFO: Logging kubelet events for node i-087c2326a312024ef Jan 9 06:44:37.916: INFO: Logging pods the kubelet thinks is on node i-087c2326a312024ef Jan 9 06:44:38.084: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-qxtwd started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.084: INFO: pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d started at 2023-01-09 06:39:05 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container write-pod ready: false, restart count 0 Jan 9 06:44:38.084: INFO: test-rs-gj54c started at 2023-01-09 06:44:23 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:38.084: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-b8q64 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.084: INFO: webserver-7f5969cbc7-rf7jj started at 2023-01-09 06:41:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:38.084: INFO: pod1 started at 2023-01-09 06:40:38 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:38.084: INFO: oidc-discovery-validator started at 2023-01-09 06:39:17 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container oidc-discovery-validator ready: false, restart count 0 Jan 9 06:44:38.084: INFO: webserver-66b8c99f6-gt295 started at 2023-01-09 06:41:30 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container httpd ready: true, restart count 0 Jan 9 06:44:38.084: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-85kn4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.084: INFO: sample-webhook-deployment-865554f4d9-sqdbc started at 2023-01-09 06:44:13 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container sample-webhook ready: false, restart count 0 Jan 9 06:44:38.084: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-dlhg8 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:44:38.084: INFO: ebs-csi-node-jvd2b started at 2023-01-09 06:25:35 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:38.084: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:38.084: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:38.084: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:38.084: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-h8zj6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.084: INFO: netserver-2 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:38.084: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pvc5d started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:44:38.084: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-r89ft started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.084: INFO: cilium-gw6mm started at 2023-01-09 06:25:35 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:38.084: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:38.084: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-plbnr started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.084: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8brjv started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.084: INFO: hostexec-i-087c2326a312024ef-2vcfq started at 2023-01-09 06:40:20 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:38.084: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pcnbb started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.084: INFO: pod-should-be-evicted95f3032f-4412-4cf0-898f-db2e0558e909 started at 2023-01-09 06:44:26 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.084: INFO: Container bar ready: false, restart count 0 Jan 9 06:44:38.636: INFO: Latency metrics for node i-087c2326a312024ef Jan 9 06:44:38.636: INFO: Logging node info for node i-0c0b18eb32e7da83d Jan 9 06:44:38.777: INFO: Node Info: &Node{ObjectMeta:{i-0c0b18eb32e7da83d f1550edf-43cd-451c-bf01-eb92d6980d45 30343 0 2023-01-09 06:25:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0c0b18eb32e7da83d kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-0c0b18eb32e7da83d topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.55.27 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0c0b18eb32e7da83d"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-09 06:44:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c0b18eb32e7da83d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.55.27,},NodeAddress{Type:ExternalIP,Address:18.228.15.186,},NodeAddress{Type:InternalDNS,Address:i-0c0b18eb32e7da83d.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0c0b18eb32e7da83d.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-15-186.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec267e00a4086801bd60f1191c81f587,SystemUUID:ec267e00-a408-6801-bd60-f1191c81f587,BootID:7c624b34-f575-4c40-8226-aef6612a1a56,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:38.778: INFO: Logging kubelet events for node i-0c0b18eb32e7da83d Jan 9 06:44:38.922: INFO: Logging pods the kubelet thinks is on node i-0c0b18eb32e7da83d Jan 9 06:44:39.088: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-ck9r4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:39.088: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-lknfk started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:39.088: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-cfps7 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:39.088: INFO: webserver-pod started at 2023-01-09 06:39:16 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container agnhost ready: false, restart count 0 Jan 9 06:44:39.088: INFO: netserver-3 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:39.088: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-njsp9 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:39.088: INFO: webserver-66b8c99f6-b9md9 started at 2023-01-09 06:44:35 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:39.088: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-mwm9d started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:39.088: INFO: ebs-csi-node-m5kg7 started at 2023-01-09 06:25:34 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:39.088: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:39.088: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:39.088: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:39.088: INFO: cilium-vx7vq started at 2023-01-09 06:25:34 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:39.088: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:39.088: INFO: coredns-559769c974-md8gz started at 2023-01-09 06:25:51 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container coredns ready: true, restart count 0 Jan 9 06:44:39.088: INFO: coredns-autoscaler-7cb5c5b969-4vwwx started at 2023-01-09 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container autoscaler ready: true, restart count 0 Jan 9 06:44:39.088: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-sf7zm started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:39.088: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-twg96 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:39.088: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-ft8kr started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:39.088: INFO: hostpath-client started at 2023-01-09 06:40:50 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container hostpath-client ready: false, restart count 0 Jan 9 06:44:39.088: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kpwrs started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:44:39.088: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kxqfw started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:39.088: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:39.645: INFO: Latency metrics for node i-0c0b18eb32e7da83d [DeferCleanup (Each)] [sig-apps] Deployment tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "deployment-7519" for this suite. �[38;5;243m01/09/23 06:44:39.645�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-auth\]\sServiceAccounts\sServiceAccountIssuerDiscovery\sshould\ssupport\sOIDC\sdiscovery\sof\sservice\saccount\sissuer\s\[Conformance\]$'
test/e2e/auth/service_accounts.go:637 k8s.io/kubernetes/test/e2e/auth.glob..func5.7() test/e2e/auth/service_accounts.go:637 +0xc92from junit_01.xml
[BeforeEach] [sig-auth] ServiceAccounts set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/09/23 06:39:16.077�[0m Jan 9 06:39:16.077: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename svcaccounts �[38;5;243m01/09/23 06:39:16.078�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/09/23 06:39:16.508�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/09/23 06:39:16.794�[0m [BeforeEach] [sig-auth] ServiceAccounts test/e2e/framework/metrics/init/init.go:31 [It] ServiceAccountIssuerDiscovery should support OIDC discovery of service account issuer [Conformance] test/e2e/auth/service_accounts.go:531 Jan 9 06:39:17.370: INFO: created pod Jan 9 06:39:17.371: INFO: Waiting up to 5m0s for pod "oidc-discovery-validator" in namespace "svcaccounts-3140" to be "Succeeded or Failed" Jan 9 06:39:17.514: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 143.131744ms Jan 9 06:39:19.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2.286669726s Jan 9 06:39:21.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4.286783897s Jan 9 06:39:23.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 6.287272201s Jan 9 06:39:25.659: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 8.288007621s Jan 9 06:39:27.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 10.286888256s Jan 9 06:39:29.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 12.286886257s Jan 9 06:39:31.660: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 14.28898508s Jan 9 06:39:33.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 16.286491527s Jan 9 06:39:35.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 18.287588835s Jan 9 06:39:37.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 20.287321228s Jan 9 06:39:39.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 22.287495236s Jan 9 06:39:41.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 24.287070485s Jan 9 06:39:43.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 26.286722873s Jan 9 06:39:45.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 28.286985945s Jan 9 06:39:47.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 30.287599643s Jan 9 06:39:49.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 32.287294245s Jan 9 06:39:51.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 34.287401312s Jan 9 06:39:53.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 36.287369606s Jan 9 06:39:55.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 38.286979582s Jan 9 06:39:57.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 40.287370302s Jan 9 06:39:59.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 42.287178661s Jan 9 06:40:01.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 44.287781888s Jan 9 06:40:03.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 46.287404654s Jan 9 06:40:05.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 48.28677437s Jan 9 06:40:07.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 50.287054516s Jan 9 06:40:09.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 52.286901954s Jan 9 06:40:11.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 54.287106406s Jan 9 06:40:13.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 56.28708555s Jan 9 06:40:15.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 58.287274317s Jan 9 06:40:17.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.287405458s Jan 9 06:40:19.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.287235751s Jan 9 06:40:21.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.287356968s Jan 9 06:40:23.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.287394722s Jan 9 06:40:25.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.28727946s Jan 9 06:40:27.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.287245175s Jan 9 06:40:29.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.286514549s Jan 9 06:40:31.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.286475285s Jan 9 06:40:33.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.286858366s Jan 9 06:40:35.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.286578529s Jan 9 06:40:37.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.286753115s Jan 9 06:40:39.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.287903771s Jan 9 06:40:41.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.286917785s Jan 9 06:40:43.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.286869635s Jan 9 06:40:45.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.286591194s Jan 9 06:40:47.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.287527206s Jan 9 06:40:49.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.286955967s Jan 9 06:40:51.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.286695316s Jan 9 06:40:53.659: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.2883172s Jan 9 06:40:55.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.286828022s Jan 9 06:40:57.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.286915091s Jan 9 06:40:59.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.287177338s Jan 9 06:41:01.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.286964337s Jan 9 06:41:03.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.286267092s Jan 9 06:41:05.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.286785151s Jan 9 06:41:07.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.287140126s Jan 9 06:41:09.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.286696565s Jan 9 06:41:11.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.286844268s Jan 9 06:41:13.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.286836073s Jan 9 06:41:15.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.287110796s Jan 9 06:41:17.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.286712101s Jan 9 06:41:19.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.286702563s Jan 9 06:41:21.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.28687842s Jan 9 06:41:23.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.286650757s Jan 9 06:41:25.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.286635779s Jan 9 06:41:27.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.287166678s Jan 9 06:41:29.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.286849327s Jan 9 06:41:31.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.286887366s Jan 9 06:41:33.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.286695726s Jan 9 06:41:35.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.287289334s Jan 9 06:41:37.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.286745722s Jan 9 06:41:39.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.286909577s Jan 9 06:41:41.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.287053689s Jan 9 06:41:43.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.287175231s Jan 9 06:41:45.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.286748766s Jan 9 06:41:47.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.286593839s Jan 9 06:41:49.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.286802742s Jan 9 06:41:51.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.287072132s Jan 9 06:41:53.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.287116462s Jan 9 06:41:55.659: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.28861546s Jan 9 06:41:57.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.287206559s Jan 9 06:41:59.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.287106664s Jan 9 06:42:01.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.287265403s Jan 9 06:42:03.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.287322029s Jan 9 06:42:05.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.286874144s Jan 9 06:42:07.663: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.292344527s Jan 9 06:42:09.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.287126714s Jan 9 06:42:11.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.287538549s Jan 9 06:42:13.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.287137807s Jan 9 06:42:15.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.287517048s Jan 9 06:42:17.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.287197876s Jan 9 06:42:19.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.286762066s Jan 9 06:42:21.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.28742226s Jan 9 06:42:23.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.286971785s Jan 9 06:42:25.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.287233028s Jan 9 06:42:27.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.287246187s Jan 9 06:42:29.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.287107049s Jan 9 06:42:31.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.287470127s Jan 9 06:42:33.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.287100052s Jan 9 06:42:35.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.287895291s Jan 9 06:42:37.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.287184855s Jan 9 06:42:39.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.287299886s Jan 9 06:42:41.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.287004798s Jan 9 06:42:43.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.28703384s Jan 9 06:42:45.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.287420909s Jan 9 06:42:47.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.287272555s Jan 9 06:42:49.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.287156945s Jan 9 06:42:51.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.287344853s Jan 9 06:42:53.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.28695174s Jan 9 06:42:55.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.287217882s Jan 9 06:42:57.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.287111111s Jan 9 06:42:59.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.287274149s Jan 9 06:43:01.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.286960743s Jan 9 06:43:03.660: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.289717562s Jan 9 06:43:05.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.2871088s Jan 9 06:43:07.659: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.288005973s Jan 9 06:43:09.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.286673074s Jan 9 06:43:11.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.287231956s Jan 9 06:43:13.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.286888375s Jan 9 06:43:15.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.287231316s Jan 9 06:43:17.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.287575544s Jan 9 06:43:19.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.28753675s Jan 9 06:43:21.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.287035262s Jan 9 06:43:23.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.286866248s Jan 9 06:43:25.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.286803022s Jan 9 06:43:27.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.287238411s Jan 9 06:43:29.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.286849378s Jan 9 06:43:31.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.287121174s Jan 9 06:43:33.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.287253465s Jan 9 06:43:35.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.287443657s Jan 9 06:43:37.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.287139548s Jan 9 06:43:39.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.286749778s Jan 9 06:43:41.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.287333038s Jan 9 06:43:43.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.286994946s Jan 9 06:43:45.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.287228051s Jan 9 06:43:47.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.287049219s Jan 9 06:43:49.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.28687232s Jan 9 06:43:51.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.286601107s Jan 9 06:43:53.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.286602564s Jan 9 06:43:55.664: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.293342799s Jan 9 06:43:57.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.287216609s Jan 9 06:43:59.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.287348978s Jan 9 06:44:01.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.287299165s Jan 9 06:44:03.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.286858628s Jan 9 06:44:05.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.287082335s Jan 9 06:44:07.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.287504201s Jan 9 06:44:09.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.287385219s Jan 9 06:44:11.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.286915277s Jan 9 06:44:13.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.287084346s Jan 9 06:44:15.657: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.286867953s Jan 9 06:44:17.658: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.287077716s Jan 9 06:44:17.801: INFO: Pod "oidc-discovery-validator": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.430385819s Jan 9 06:44:47.802: INFO: polling logs Jan 9 06:44:47.949: INFO: Pod logs: I0109 06:44:38.165808 1 log.go:198] OK: Got token I0109 06:44:38.165852 1 log.go:198] validating with in-cluster discovery I0109 06:44:38.166162 1 log.go:198] OK: got issuer https://api.internal.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io I0109 06:44:38.166198 1 log.go:198] Full, not-validated claims: openidmetadata.claims{Claims:jwt.Claims{Issuer:"https://api.internal.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io", Subject:"system:serviceaccount:svcaccounts-3140:default", Audience:jwt.Audience{"oidc-discovery-test"}, Expiry:1673246957, NotBefore:1673246357, IssuedAt:1673246357, ID:""}, Kubernetes:openidmetadata.kubeClaims{Namespace:"svcaccounts-3140", ServiceAccount:openidmetadata.kubeName{Name:"default", UID:"d9078a51-b0f9-485d-9312-f4f76edfc3c2"}}} I0109 06:44:38.179696 1 log.go:198] OK: Constructed OIDC provider for issuer https://api.internal.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io I0109 06:44:38.182066 1 log.go:198] OK: Validated signature on JWT I0109 06:44:38.182138 1 log.go:198] OK: Got valid claims from token! I0109 06:44:38.182157 1 log.go:198] Full, validated claims: &openidmetadata.claims{Claims:jwt.Claims{Issuer:"https://api.internal.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io", Subject:"system:serviceaccount:svcaccounts-3140:default", Audience:jwt.Audience{"oidc-discovery-test"}, Expiry:1673246957, NotBefore:1673246357, IssuedAt:1673246357, ID:""}, Kubernetes:openidmetadata.kubeClaims{Namespace:"svcaccounts-3140", ServiceAccount:openidmetadata.kubeName{Name:"default", UID:"d9078a51-b0f9-485d-9312-f4f76edfc3c2"}}} Jan 9 06:44:47.950: INFO: Unexpected error: <*pod.timeoutError | 0xc000bd49c0>: { msg: "timed out while waiting for pod svcaccounts-3140/oidc-discovery-validator to be Succeeded or Failed", observedObjects: [ <*v1.Pod | 0xc00479cd80>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "oidc-discovery-validator", GenerateName: "", Namespace: "svcaccounts-3140", SelfLink: "", UID: "1a8a002e-d5b0-4fb0-99a3-899247a3bfed", ResourceVersion: "27938", Generation: 0, CreationTimestamp: { Time: { wall: 0, ext: 63808843157, loc: { name: "Local", zone: [ {name: "UTC", offset: 0, isDST: false}, ], tx: [ { when: -576460752303423488, index: 0, isstd: false, isutc: false, }, ], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: "UTC", offset: 0, isDST: false}, }, }, }, DeletionTimestamp: nil, DeletionGracePeriodSeconds: nil, Labels: nil, Annotations: nil, OwnerReferences: nil, Finalizers: nil, ManagedFields: [ { Manager: "e2e.test", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63808843157, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"oidc-discovery-validator\\\"}\":{\".\":{},\"f:args\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{},\"f:volumeMounts\":{\".\":{},\"k:{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\"}\":{\".\":{},\"f:mountPath\":{},\"f:name\":{},\"f:readOnly\":{}}}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:serviceAccount\":{},\"f:serviceAccountName\":{},\"f:terminationGracePeriodSeconds\":{},\"f:volumes\":{\".\":{},\"k:{\\\"name\\\":\\\"sa-token\\\"}\":{\".\":{},\"f:name\":{},\"f:projected\":{\".\":{},\"f:defaultMode\":{},\"f:sources\":{}}}}}}", }, Subresource: "", }, { Manager: "kubelet", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63808843157, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}", }, Subresource: "status", }, ], }, Spec: { Volumes: [ { Name: "sa-token", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: nil, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: { Sources: [ { Secret: ..., DownwardAPI: ..., ConfigMap: ..., ServiceAccountToken: ..., }, ], DefaultMode: 420, }, PortworxVolume: nil, ScaleIO: nil, StorageOS: nil, CSI: nil, Ephemeral: nil, }, }, { Name: "kube-api-access-qjsts", VolumeSource: { Host... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Jan 9 06:44:47.950: FAIL: timed out while waiting for pod svcaccounts-3140/oidc-discovery-validator to be Succeeded or Failed Full Stack Trace k8s.io/kubernetes/test/e2e/auth.glob..func5.7() test/e2e/auth/service_accounts.go:637 +0xc92 [AfterEach] [sig-auth] ServiceAccounts test/e2e/framework/node/init/init.go:32 Jan 9 06:44:48.094: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-auth] ServiceAccounts test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-auth] ServiceAccounts dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/09/23 06:44:48.238�[0m �[1mSTEP:�[0m Collecting events from namespace "svcaccounts-3140". �[38;5;243m01/09/23 06:44:48.239�[0m �[1mSTEP:�[0m Found 13 events. �[38;5;243m01/09/23 06:44:48.382�[0m Jan 9 06:44:48.382: INFO: At 2023-01-09 06:39:17 +0000 UTC - event for oidc-discovery-validator: {default-scheduler } Scheduled: Successfully assigned svcaccounts-3140/oidc-discovery-validator to i-087c2326a312024ef Jan 9 06:44:48.382: INFO: At 2023-01-09 06:39:17 +0000 UTC - event for oidc-discovery-validator: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f3d83caa3a53f77d0dc66f74cfa535ff11603c96f9228210391f83573c996092": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:48.382: INFO: At 2023-01-09 06:39:32 +0000 UTC - event for oidc-discovery-validator: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b778bc679b3dcc9154096a5e152f9a5f502d84863dc624befe724a5eae5d2bd9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:48.382: INFO: At 2023-01-09 06:39:46 +0000 UTC - event for oidc-discovery-validator: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f8d9834476cbb47c0c33ef118e2ad7a11cf63656d5f7b92e7c151ed75b16ad9c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:48.382: INFO: At 2023-01-09 06:40:01 +0000 UTC - event for oidc-discovery-validator: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ac770d1bd9bc0f5c9a61ea71796dd5a8539b5de2d5d5a1fdad84f502988c0abd": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:48.382: INFO: At 2023-01-09 06:40:12 +0000 UTC - event for oidc-discovery-validator: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "463dbd1c9e1ce51c173a41414514f7ead7a66f601216dd1a8bf09f4aeae310ca": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:48.382: INFO: At 2023-01-09 06:40:23 +0000 UTC - event for oidc-discovery-validator: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "05d47e8a12bb5c07498d432b5b31769ffe92d15fd97ee58c561253b700b0c810": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:48.382: INFO: At 2023-01-09 06:40:35 +0000 UTC - event for oidc-discovery-validator: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "de6812ec6e7b1eca4de99f713b011d543143955583651088161d8f58931526f1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:48.382: INFO: At 2023-01-09 06:40:47 +0000 UTC - event for oidc-discovery-validator: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e84d48bfea7e18c35cb930a6c0f6bb697d5bf5fd8bb06b5bbd05acb190899d90": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:48.382: INFO: At 2023-01-09 06:41:02 +0000 UTC - event for oidc-discovery-validator: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d92d01f5c1fe29dd35adb0a192dfec79d8d2f62e39dfd41ebdbf23833defb75a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:48.382: INFO: At 2023-01-09 06:41:16 +0000 UTC - event for oidc-discovery-validator: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e916761150a8d32f0d30922fecdd809d4b2da932fc10127eb3b2f5ce8d4ad96c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:48.382: INFO: At 2023-01-09 06:44:38 +0000 UTC - event for oidc-discovery-validator: {kubelet i-087c2326a312024ef} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Jan 9 06:44:48.382: INFO: At 2023-01-09 06:44:38 +0000 UTC - event for oidc-discovery-validator: {kubelet i-087c2326a312024ef} Created: Created container oidc-discovery-validator Jan 9 06:44:48.525: INFO: POD NODE PHASE GRACE CONDITIONS Jan 9 06:44:48.525: INFO: oidc-discovery-validator i-087c2326a312024ef Succeeded [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:17 +0000 UTC PodCompleted } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:17 +0000 UTC PodCompleted } {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:17 +0000 UTC PodCompleted } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:17 +0000 UTC }] Jan 9 06:44:48.525: INFO: Jan 9 06:44:48.814: INFO: Logging node info for node i-02b09d19b2096261b Jan 9 06:44:48.957: INFO: Node Info: &Node{ObjectMeta:{i-02b09d19b2096261b efa11f02-6d54-4902-bbeb-efecc1f02cd1 29640 0 2023-01-09 06:23:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-02b09d19b2096261b kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.12 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02b09d19b2096261b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-09 06:23:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-09 06:24:22 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-09 06:24:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-09 06:41:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-02b09d19b2096261b,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:24:43 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.12,},NodeAddress{Type:ExternalIP,Address:54.94.119.205,},NodeAddress{Type:InternalDNS,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-119-205.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2da51bd0e40b36975082ac186a369f,SystemUUID:ec2da51b-d0e4-0b36-9750-82ac186a369f,BootID:83891f80-d7aa-4da9-ab10-44706ab3b631,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43453930,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42803458,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967347,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:48.957: INFO: Logging kubelet events for node i-02b09d19b2096261b Jan 9 06:44:49.102: INFO: Logging pods the kubelet thinks is on node i-02b09d19b2096261b Jan 9 06:44:49.251: INFO: etcd-manager-main-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:49.251: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:44:49.251: INFO: kube-apiserver-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+2 container statuses recorded) Jan 9 06:44:49.251: INFO: Container healthcheck ready: true, restart count 0 Jan 9 06:44:49.251: INFO: Container kube-apiserver ready: true, restart count 1 Jan 9 06:44:49.251: INFO: cilium-97g4h started at 2023-01-09 06:24:28 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:49.251: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:49.251: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:49.251: INFO: dns-controller-69987775c6-mv7h2 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:49.251: INFO: Container dns-controller ready: true, restart count 0 Jan 9 06:44:49.251: INFO: cilium-operator-5dd44dc49f-fgc7c started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:49.251: INFO: Container cilium-operator ready: true, restart count 0 Jan 9 06:44:49.251: INFO: kube-controller-manager-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:49.251: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 9 06:44:49.251: INFO: kube-scheduler-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:49.251: INFO: Container kube-scheduler ready: true, restart count 0 Jan 9 06:44:49.251: INFO: etcd-manager-events-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:49.251: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:44:49.251: INFO: aws-cloud-controller-manager-kj4b6 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:49.251: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 9 06:44:49.251: INFO: kops-controller-8qjsv started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:49.251: INFO: Container kops-controller ready: true, restart count 0 Jan 9 06:44:49.251: INFO: ebs-csi-node-kblmh started at 2023-01-09 06:24:28 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:49.251: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:49.251: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:49.251: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:49.251: INFO: ebs-csi-controller-89b649c94-zwk75 started at 2023-01-09 06:24:28 +0000 UTC (0+5 container statuses recorded) Jan 9 06:44:49.251: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:44:49.251: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:44:49.251: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:44:49.251: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:49.251: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:49.751: INFO: Latency metrics for node i-02b09d19b2096261b Jan 9 06:44:49.751: INFO: Logging node info for node i-0585809501bfc393d Jan 9 06:44:49.894: INFO: Node Info: &Node{ObjectMeta:{i-0585809501bfc393d 8c7c1c77-e673-43a0-81a4-fcf97493bcd8 29271 0 2023-01-09 06:25:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0585809501bfc393d kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-0585809501bfc393d topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.147 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0585809501bfc393d"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-09 06:41:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0585809501bfc393d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.147,},NodeAddress{Type:ExternalIP,Address:54.94.242.160,},NodeAddress{Type:InternalDNS,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-242-160.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec206fc38cf93b7ea12ba838ad958aa3,SystemUUID:ec206fc3-8cf9-3b7e-a12b-a838ad958aa3,BootID:afb66846-6976-49bf-9e61-b54e5af8a3eb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:49.894: INFO: Logging kubelet events for node i-0585809501bfc393d Jan 9 06:44:50.040: INFO: Logging pods the kubelet thinks is on node i-0585809501bfc393d Jan 9 06:44:50.191: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-bxrs7 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:50.191: INFO: netserver-0 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:50.191: INFO: cilium-f5j8w started at 2023-01-09 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:50.191: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:50.191: INFO: pod-secrets-2e68e54b-1a9c-4316-a873-c553608b50e7 started at 2023-01-09 06:44:42 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container secret-test ready: false, restart count 0 Jan 9 06:44:50.191: INFO: dns-test-bdcfcfaa-2367-4a35-bb69-c5dd9d18f66c started at 2023-01-09 06:44:29 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:50.191: INFO: Container jessie-querier ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container querier ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:50.191: INFO: httpd-deployment-6859889f88-d97t4 started at 2023-01-09 06:44:43 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:50.191: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fs9d6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:50.191: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:39:17 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:50.191: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:50.191: INFO: ebs-csi-node-6djgn started at 2023-01-09 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:50.191: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:50.191: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:50.191: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:50.191: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fm99c started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:50.191: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-xtk85 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:50.191: INFO: alpine-nnp-false-4247b683-138d-4632-a6a3-6d844097c353 started at 2023-01-09 06:41:15 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container alpine-nnp-false-4247b683-138d-4632-a6a3-6d844097c353 ready: false, restart count 0 Jan 9 06:44:50.191: INFO: webserver-66b8c99f6-ls7f4 started at 2023-01-09 06:41:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:50.191: INFO: execpodjqtvg started at 2023-01-09 06:42:07 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container agnhost-container ready: false, restart count 0 Jan 9 06:44:50.191: INFO: hostexec-i-0585809501bfc393d-7kwvv started at 2023-01-09 06:44:39 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:50.191: INFO: dns-test-a96d67de-eeb2-4e87-8d66-17c313a4ef5a started at 2023-01-09 06:40:06 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:50.191: INFO: Container jessie-querier ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container querier ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:50.191: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-qm6jm started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:50.191: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8rq57 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:50.191: INFO: httpd-deployment-7b78564759-vv2ps started at 2023-01-09 06:44:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:50.191: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-2jgj6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:50.191: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-6hzsb started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:50.191: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:40:25 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:50.191: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:50.191: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:50.191: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-cs2m5 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:50.191: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-97f28 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:50.191: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-p6pfg started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:50.191: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:50.752: INFO: Latency metrics for node i-0585809501bfc393d Jan 9 06:44:50.752: INFO: Logging node info for node i-06d5855698aa7cb08 Jan 9 06:44:50.896: INFO: Node Info: &Node{ObjectMeta:{i-06d5855698aa7cb08 bdfd8f9d-5aa1-4cff-806a-849d1c29fcb6 30894 0 2023-01-09 06:25:34 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06d5855698aa7cb08 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-06d5855698aa7cb08 topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.46.163 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-2682":"i-06d5855698aa7cb08","ebs.csi.aws.com":"i-06d5855698aa7cb08"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-09 06:44:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06d5855698aa7cb08,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:48 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:48 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:48 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:44:48 +0000 UTC,LastTransitionTime:2023-01-09 06:25:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.46.163,},NodeAddress{Type:ExternalIP,Address:18.228.235.107,},NodeAddress{Type:InternalDNS,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-107.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29e10a0890d67138c4eff5b197ee61,SystemUUID:ec29e10a-0890-d671-38c4-eff5b197ee61,BootID:9352599c-af91-4c75-860f-bb9a2789a998,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:50.896: INFO: Logging kubelet events for node i-06d5855698aa7cb08 Jan 9 06:44:51.041: INFO: Logging pods the kubelet thinks is on node i-06d5855698aa7cb08 Jan 9 06:44:51.192: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-mkn6f started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:51.192: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:51.192: INFO: webserver-7f5969cbc7-tz85c started at 2023-01-09 06:41:16 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:51.192: INFO: Container httpd ready: true, restart count 0 Jan 9 06:44:51.192: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-vwlp2 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:51.192: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:51.192: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pnbg6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:51.192: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:51.192: INFO: ebs-csi-node-6dzh8 started at 2023-01-09 06:25:35 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:51.192: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:51.192: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:51.192: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:51.192: INFO: netserver-1 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:51.192: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:51.192: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:39:00 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:51.192: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:44:51.192: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:44:51.192: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:44:51.192: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 9 06:44:51.192: INFO: Container hostpath ready: true, restart count 0 Jan 9 06:44:51.192: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:51.192: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:51.192: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:42:19 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:51.192: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:51.192: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:51.192: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:51.192: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:51.192: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:51.192: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:51.192: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:51.192: INFO: httpd-deployment-6859889f88-stp8n started at 2023-01-09 06:44:45 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:51.192: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:51.192: INFO: coredns-559769c974-qdhtp started at 2023-01-09 06:26:21 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:51.192: INFO: Container coredns ready: true, restart count 0 Jan 9 06:44:51.192: INFO: csi-mockplugin-0 started at 2023-01-09 06:40:46 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:51.192: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:51.192: INFO: Container driver-registrar ready: false, restart count 0 Jan 9 06:44:51.192: INFO: Container mock ready: false, restart count 0 Jan 9 06:44:51.192: INFO: cilium-wbn8d started at 2023-01-09 06:25:35 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:51.192: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:51.192: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:51.192: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fncl4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:51.192: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:51.192: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8hxw6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:51.192: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:51.192: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-wg2gn started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:51.192: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:51.192: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-hpb4w started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:51.192: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:51.192: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kr2c4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:51.192: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:51.192: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-b42jp started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:51.192: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:51.813: INFO: Latency metrics for node i-06d5855698aa7cb08 Jan 9 06:44:51.813: INFO: Logging node info for node i-087c2326a312024ef Jan 9 06:44:51.957: INFO: Node Info: &Node{ObjectMeta:{i-087c2326a312024ef 45db8cc8-250b-411a-b344-d6bbffe46b7d 29314 0 2023-01-09 06:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-087c2326a312024ef kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-087c2326a312024ef topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.218 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-087c2326a312024ef"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:40:12 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-09 06:41:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-087c2326a312024ef,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.218,},NodeAddress{Type:ExternalIP,Address:18.230.17.62,},NodeAddress{Type:InternalDNS,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-17-62.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec28fbdf9f29233021716214b9f8da3a,SystemUUID:ec28fbdf-9f29-2330-2171-6214b9f8da3a,BootID:4dbf8c75-8514-419f-b443-1a44f89f420f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d49342c84d2a787d],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d49342c84d2a787d,DevicePath:,},},Config:nil,},} Jan 9 06:44:51.958: INFO: Logging kubelet events for node i-087c2326a312024ef Jan 9 06:44:52.103: INFO: Logging pods the kubelet thinks is on node i-087c2326a312024ef Jan 9 06:44:52.264: INFO: sample-webhook-deployment-865554f4d9-sqdbc started at 2023-01-09 06:44:13 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.264: INFO: Container sample-webhook ready: false, restart count 0 Jan 9 06:44:52.265: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-dlhg8 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:44:52.265: INFO: ebs-csi-node-jvd2b started at 2023-01-09 06:25:35 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:52.265: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:52.265: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:52.265: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:52.265: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-h8zj6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:52.265: INFO: netserver-2 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:52.265: INFO: cilium-gw6mm started at 2023-01-09 06:25:35 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:52.265: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:52.265: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-plbnr started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:52.265: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pvc5d started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:44:52.265: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-r89ft started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:52.265: INFO: hostexec-i-087c2326a312024ef-srclt started at 2023-01-09 06:44:43 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:52.265: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8brjv started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:52.265: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pcnbb started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:52.265: INFO: pod-should-be-evicted95f3032f-4412-4cf0-898f-db2e0558e909 started at 2023-01-09 06:44:26 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container bar ready: false, restart count 0 Jan 9 06:44:52.265: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-qxtwd started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:52.265: INFO: pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d started at 2023-01-09 06:39:05 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container write-pod ready: true, restart count 0 Jan 9 06:44:52.265: INFO: test-rs-gj54c started at 2023-01-09 06:44:23 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:52.265: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-b8q64 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:52.265: INFO: pod1 started at 2023-01-09 06:40:38 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:52.265: INFO: oidc-discovery-validator started at 2023-01-09 06:39:17 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container oidc-discovery-validator ready: false, restart count 0 Jan 9 06:44:52.265: INFO: webserver-66b8c99f6-gt295 started at 2023-01-09 06:41:30 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container httpd ready: true, restart count 0 Jan 9 06:44:52.265: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-85kn4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:52.265: INFO: httpd-deployment-6859889f88-dcwbf started at 2023-01-09 06:44:42 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:52.265: INFO: webserver-7f5969cbc7-rf7jj started at 2023-01-09 06:41:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:52.265: INFO: var-expansion-a2239f26-e74b-4412-8ed1-edb727aba048 started at 2023-01-09 06:44:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:52.265: INFO: Container dapi-container ready: false, restart count 0 Jan 9 06:44:52.777: INFO: Latency metrics for node i-087c2326a312024ef Jan 9 06:44:52.777: INFO: Logging node info for node i-0c0b18eb32e7da83d Jan 9 06:44:52.921: INFO: Node Info: &Node{ObjectMeta:{i-0c0b18eb32e7da83d f1550edf-43cd-451c-bf01-eb92d6980d45 30343 0 2023-01-09 06:25:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0c0b18eb32e7da83d kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-0c0b18eb32e7da83d topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.55.27 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0c0b18eb32e7da83d"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-09 06:44:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c0b18eb32e7da83d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.55.27,},NodeAddress{Type:ExternalIP,Address:18.228.15.186,},NodeAddress{Type:InternalDNS,Address:i-0c0b18eb32e7da83d.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0c0b18eb32e7da83d.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-15-186.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec267e00a4086801bd60f1191c81f587,SystemUUID:ec267e00-a408-6801-bd60-f1191c81f587,BootID:7c624b34-f575-4c40-8226-aef6612a1a56,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:52.921: INFO: Logging kubelet events for node i-0c0b18eb32e7da83d Jan 9 06:44:53.066: INFO: Logging pods the kubelet thinks is on node i-0c0b18eb32e7da83d Jan 9 06:44:53.215: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-ft8kr started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:53.215: INFO: hostpath-client started at 2023-01-09 06:40:50 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container hostpath-client ready: false, restart count 0 Jan 9 06:44:53.215: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kpwrs started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:44:53.215: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kxqfw started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:53.215: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-ck9r4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:53.215: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-lknfk started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:53.215: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-cfps7 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:53.215: INFO: webserver-pod started at 2023-01-09 06:39:16 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container agnhost ready: false, restart count 0 Jan 9 06:44:53.215: INFO: netserver-3 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:53.215: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-njsp9 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:53.215: INFO: webserver-66b8c99f6-b9md9 started at 2023-01-09 06:44:35 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:53.215: INFO: ebs-csi-node-m5kg7 started at 2023-01-09 06:25:34 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:53.215: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:53.215: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:53.215: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:53.215: INFO: cilium-vx7vq started at 2023-01-09 06:25:34 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:53.215: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:53.215: INFO: coredns-559769c974-md8gz started at 2023-01-09 06:25:51 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container coredns ready: true, restart count 0 Jan 9 06:44:53.215: INFO: coredns-autoscaler-7cb5c5b969-4vwwx started at 2023-01-09 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container autoscaler ready: true, restart count 0 Jan 9 06:44:53.215: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-sf7zm started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:53.215: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-mwm9d started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:53.215: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-twg96 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:53.215: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:53.789: INFO: Latency metrics for node i-0c0b18eb32e7da83d [DeferCleanup (Each)] [sig-auth] ServiceAccounts tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "svcaccounts-3140" for this suite. �[38;5;243m01/09/23 06:44:53.789�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-auth\]\sServiceAccounts\sshould\sset\sownership\sand\spermission\swhen\sRunAsUser\sor\sFsGroup\sis\spresent\s\[LinuxOnly\]\s\[NodeFeature\:FSGroup\]$'
test/e2e/framework/pod/output/output.go:237 k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutputMatcher(0x760879d?, {0x7616b36?, 0xc0036a3e08?}, 0xc0046fcd80, 0x0, {0xc0036a3e88, 0x4, 0x4}, 0x4?) test/e2e/framework/pod/output/output.go:237 +0x176 k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutputRegexp(...) test/e2e/framework/pod/output/output.go:221 k8s.io/kubernetes/test/e2e/auth.glob..func5.5() test/e2e/auth/service_accounts.go:428 +0x9b5from junit_01.xml
[BeforeEach] [sig-auth] ServiceAccounts set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/09/23 06:38:56.926�[0m Jan 9 06:38:56.926: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename svcaccounts �[38;5;243m01/09/23 06:38:56.927�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/09/23 06:38:57.539�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/09/23 06:38:57.828�[0m [BeforeEach] [sig-auth] ServiceAccounts test/e2e/framework/metrics/init/init.go:31 [It] should set ownership and permission when RunAsUser or FsGroup is present [LinuxOnly] [NodeFeature:FSGroup] test/e2e/auth/service_accounts.go:336 �[1mSTEP:�[0m Creating a pod to test service account token: �[38;5;243m01/09/23 06:38:58.119�[0m Jan 9 06:38:58.265: INFO: Waiting up to 5m0s for pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db" in namespace "svcaccounts-1924" to be "Succeeded or Failed" Jan 9 06:38:58.407: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 142.122837ms Jan 9 06:39:00.560: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2.295228127s Jan 9 06:39:02.551: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4.285665097s Jan 9 06:39:04.550: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 6.285043722s Jan 9 06:39:06.550: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 8.284955041s Jan 9 06:39:08.550: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Succeeded", Reason="", readiness=false. Elapsed: 10.285464737s �[1mSTEP:�[0m Saw pod success �[38;5;243m01/09/23 06:39:08.55�[0m Jan 9 06:39:08.551: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db" satisfied condition "Succeeded or Failed" Jan 9 06:39:08.693: INFO: Trying to get logs from node i-0c0b18eb32e7da83d pod test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db container agnhost-container: <nil> �[1mSTEP:�[0m delete the pod �[38;5;243m01/09/23 06:39:08.837�[0m Jan 9 06:39:08.987: INFO: Waiting for pod test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db to disappear Jan 9 06:39:09.130: INFO: Pod test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db no longer exists �[1mSTEP:�[0m Creating a pod to test service account token: �[38;5;243m01/09/23 06:39:09.13�[0m Jan 9 06:39:09.276: INFO: Waiting up to 5m0s for pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db" in namespace "svcaccounts-1924" to be "Succeeded or Failed" Jan 9 06:39:09.418: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 142.32848ms Jan 9 06:39:11.568: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2.291803076s Jan 9 06:39:13.563: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4.286740545s Jan 9 06:39:15.562: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 6.285718709s Jan 9 06:39:17.561: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 8.285207254s Jan 9 06:39:19.562: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 10.28586358s Jan 9 06:39:21.567: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 12.290716892s Jan 9 06:39:23.561: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 14.284692194s Jan 9 06:39:25.561: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 16.284847147s Jan 9 06:39:27.562: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 18.285668157s Jan 9 06:39:29.562: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Running", Reason="", readiness=true. Elapsed: 20.285689108s Jan 9 06:39:31.561: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Succeeded", Reason="", readiness=false. Elapsed: 22.284749758s �[1mSTEP:�[0m Saw pod success �[38;5;243m01/09/23 06:39:31.561�[0m Jan 9 06:39:31.561: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db" satisfied condition "Succeeded or Failed" Jan 9 06:39:31.707: INFO: Trying to get logs from node i-0585809501bfc393d pod test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db container agnhost-container: <nil> �[1mSTEP:�[0m delete the pod �[38;5;243m01/09/23 06:39:32.144�[0m Jan 9 06:39:32.295: INFO: Waiting for pod test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db to disappear Jan 9 06:39:32.440: INFO: Pod test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db no longer exists �[1mSTEP:�[0m Creating a pod to test service account token: �[38;5;243m01/09/23 06:39:32.44�[0m Jan 9 06:39:32.586: INFO: Waiting up to 5m0s for pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db" in namespace "svcaccounts-1924" to be "Succeeded or Failed" Jan 9 06:39:32.728: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 142.088752ms Jan 9 06:39:34.875: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2.28902307s Jan 9 06:39:36.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4.285700154s Jan 9 06:39:38.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 6.285464808s Jan 9 06:39:40.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 8.285414132s Jan 9 06:39:42.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 10.284759357s Jan 9 06:39:44.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 12.285421084s Jan 9 06:39:46.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 14.286148539s Jan 9 06:39:48.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 16.285576762s Jan 9 06:39:50.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 18.284600137s Jan 9 06:39:52.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 20.285323767s Jan 9 06:39:54.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 22.285403812s Jan 9 06:39:56.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 24.285483499s Jan 9 06:39:58.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 26.284910457s Jan 9 06:40:00.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 28.285204707s Jan 9 06:40:02.873: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 30.286563547s Jan 9 06:40:04.873: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 32.286783651s Jan 9 06:40:06.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 34.285195034s Jan 9 06:40:08.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 36.28537659s Jan 9 06:40:10.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 38.285798073s Jan 9 06:40:12.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 40.285442407s Jan 9 06:40:14.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 42.285133454s Jan 9 06:40:16.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 44.285298624s Jan 9 06:40:18.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 46.284817775s Jan 9 06:40:20.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 48.285806421s Jan 9 06:40:22.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 50.28440295s Jan 9 06:40:24.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 52.285545091s Jan 9 06:40:26.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 54.284933397s Jan 9 06:40:28.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 56.285538637s Jan 9 06:40:30.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 58.285224821s Jan 9 06:40:32.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.285508698s Jan 9 06:40:34.873: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.286724607s Jan 9 06:40:36.874: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.287347195s Jan 9 06:40:38.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.284706371s Jan 9 06:40:40.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.284807325s Jan 9 06:40:42.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.28501398s Jan 9 06:40:44.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.285400709s Jan 9 06:40:46.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.285223981s Jan 9 06:40:48.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.284725736s Jan 9 06:40:50.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.28567526s Jan 9 06:40:52.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.285526645s Jan 9 06:40:54.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.284853276s Jan 9 06:40:56.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.285078295s Jan 9 06:40:58.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.285691363s Jan 9 06:41:00.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.28533271s Jan 9 06:41:02.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.285673887s Jan 9 06:41:04.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.285197846s Jan 9 06:41:06.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.285762096s Jan 9 06:41:08.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.285398861s Jan 9 06:41:10.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.285916888s Jan 9 06:41:12.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.28547959s Jan 9 06:41:14.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.28483988s Jan 9 06:41:16.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.285104667s Jan 9 06:41:18.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.285412438s Jan 9 06:41:20.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.284465794s Jan 9 06:41:22.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.285522589s Jan 9 06:41:24.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.285590625s Jan 9 06:41:26.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.285840557s Jan 9 06:41:28.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.284869659s Jan 9 06:41:30.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.285571587s Jan 9 06:41:32.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.285606456s Jan 9 06:41:34.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.28588976s Jan 9 06:41:36.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.285272742s Jan 9 06:41:38.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.285615224s Jan 9 06:41:40.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.286031387s Jan 9 06:41:42.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.285925804s Jan 9 06:41:44.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.285382289s Jan 9 06:41:46.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.284495315s Jan 9 06:41:48.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.285251782s Jan 9 06:41:50.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.285029596s Jan 9 06:41:52.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.28551726s Jan 9 06:41:54.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.285939366s Jan 9 06:41:56.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.285390142s Jan 9 06:41:58.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.285182747s Jan 9 06:42:00.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.28544938s Jan 9 06:42:02.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.285558595s Jan 9 06:42:04.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.286134223s Jan 9 06:42:06.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.2852621s Jan 9 06:42:08.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.286264667s Jan 9 06:42:10.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.285516611s Jan 9 06:42:12.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.285487003s Jan 9 06:42:14.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.285575772s Jan 9 06:42:16.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.285612789s Jan 9 06:42:18.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.284702561s Jan 9 06:42:20.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.285051596s Jan 9 06:42:22.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.28558338s Jan 9 06:42:24.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.285453613s Jan 9 06:42:26.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.285555894s Jan 9 06:42:28.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.285276247s Jan 9 06:42:30.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.284862515s Jan 9 06:42:32.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.285364223s Jan 9 06:42:34.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.284562629s Jan 9 06:42:36.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.285819985s Jan 9 06:42:38.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.285918193s Jan 9 06:42:40.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.285958517s Jan 9 06:42:42.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.285634582s Jan 9 06:42:44.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.285369716s Jan 9 06:42:46.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.285712764s Jan 9 06:42:48.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.285603249s Jan 9 06:42:50.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.286238373s Jan 9 06:42:52.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.285796872s Jan 9 06:42:54.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.285471843s Jan 9 06:42:56.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.285760645s Jan 9 06:42:58.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.285590284s Jan 9 06:43:00.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.285681802s Jan 9 06:43:02.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.28595064s Jan 9 06:43:04.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.285575006s Jan 9 06:43:06.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.285614586s Jan 9 06:43:08.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.285738502s Jan 9 06:43:10.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.285521822s Jan 9 06:43:12.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.285805348s Jan 9 06:43:14.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.285033559s Jan 9 06:43:16.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.285297979s Jan 9 06:43:18.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.285655227s Jan 9 06:43:20.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.285341323s Jan 9 06:43:22.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.285495917s Jan 9 06:43:24.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.285722851s Jan 9 06:43:26.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.284622733s Jan 9 06:43:28.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.28486721s Jan 9 06:43:30.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.285150815s Jan 9 06:43:32.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.285485079s Jan 9 06:43:34.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.285855844s Jan 9 06:43:36.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.285333139s Jan 9 06:43:38.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.285496045s Jan 9 06:43:40.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.284787854s Jan 9 06:43:42.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.285712591s Jan 9 06:43:44.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.285694473s Jan 9 06:43:46.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.285556958s Jan 9 06:43:48.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.284835783s Jan 9 06:43:50.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.285295487s Jan 9 06:43:52.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.284986101s Jan 9 06:43:54.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.285815869s Jan 9 06:43:56.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.285397334s Jan 9 06:43:58.873: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.286728026s Jan 9 06:44:00.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.28549448s Jan 9 06:44:02.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.285721505s Jan 9 06:44:04.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.285592383s Jan 9 06:44:06.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.285490369s Jan 9 06:44:08.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.284671898s Jan 9 06:44:10.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.284788964s Jan 9 06:44:12.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.284590655s Jan 9 06:44:14.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.284479998s Jan 9 06:44:16.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.285092409s Jan 9 06:44:18.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.285646443s Jan 9 06:44:20.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.284577852s Jan 9 06:44:22.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.28572082s Jan 9 06:44:24.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.285667967s Jan 9 06:44:26.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.285368142s Jan 9 06:44:28.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.285669329s Jan 9 06:44:30.871: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.284862963s Jan 9 06:44:32.872: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.285741018s Jan 9 06:44:33.014: INFO: Pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.428184341s Jan 9 06:44:33.300: INFO: Failed to get logs from node "i-087c2326a312024ef" pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db" container "agnhost-container": the server rejected our request for an unknown reason (get pods test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db) �[1mSTEP:�[0m delete the pod �[38;5;243m01/09/23 06:44:33.3�[0m Jan 9 06:44:33.450: INFO: Waiting for pod test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db to disappear Jan 9 06:44:33.592: INFO: Pod test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db no longer exists Jan 9 06:44:33.593: INFO: Unexpected error: <*errors.errorString | 0xc000be89e0>: { s: "expected pod \"test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db\" success: timed out while waiting for pod svcaccounts-1924/test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db to be Succeeded or Failed", } Jan 9 06:44:33.593: FAIL: expected pod "test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db" success: timed out while waiting for pod svcaccounts-1924/test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db to be Succeeded or Failed Full Stack Trace k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutputMatcher(0x760879d?, {0x7616b36?, 0xc0036a3e08?}, 0xc0046fcd80, 0x0, {0xc0036a3e88, 0x4, 0x4}, 0x4?) test/e2e/framework/pod/output/output.go:237 +0x176 k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutputRegexp(...) test/e2e/framework/pod/output/output.go:221 k8s.io/kubernetes/test/e2e/auth.glob..func5.5() test/e2e/auth/service_accounts.go:428 +0x9b5 [AfterEach] [sig-auth] ServiceAccounts test/e2e/framework/node/init/init.go:32 Jan 9 06:44:33.593: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-auth] ServiceAccounts test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-auth] ServiceAccounts dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/09/23 06:44:33.736�[0m �[1mSTEP:�[0m Collecting events from namespace "svcaccounts-1924". �[38;5;243m01/09/23 06:44:33.736�[0m �[1mSTEP:�[0m Found 20 events. �[38;5;243m01/09/23 06:44:33.879�[0m Jan 9 06:44:33.879: INFO: At 2023-01-09 06:38:58 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {default-scheduler } Scheduled: Successfully assigned svcaccounts-1924/test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db to i-0c0b18eb32e7da83d Jan 9 06:44:33.879: INFO: At 2023-01-09 06:38:58 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-0c0b18eb32e7da83d} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Jan 9 06:44:33.879: INFO: At 2023-01-09 06:38:58 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-0c0b18eb32e7da83d} Created: Created container agnhost-container Jan 9 06:44:33.879: INFO: At 2023-01-09 06:38:59 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-0c0b18eb32e7da83d} Started: Started container agnhost-container Jan 9 06:44:33.879: INFO: At 2023-01-09 06:39:09 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {default-scheduler } Scheduled: Successfully assigned svcaccounts-1924/test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db to i-0585809501bfc393d Jan 9 06:44:33.879: INFO: At 2023-01-09 06:39:09 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bd62f8082851dc19115012dc6aa04fad7a82200a3de4bba9cbcdbe4b27bb677d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:33.879: INFO: At 2023-01-09 06:39:23 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-0585809501bfc393d} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Jan 9 06:44:33.879: INFO: At 2023-01-09 06:39:23 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-0585809501bfc393d} Created: Created container agnhost-container Jan 9 06:44:33.879: INFO: At 2023-01-09 06:39:23 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-0585809501bfc393d} Started: Started container agnhost-container Jan 9 06:44:33.879: INFO: At 2023-01-09 06:39:32 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {default-scheduler } Scheduled: Successfully assigned svcaccounts-1924/test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db to i-087c2326a312024ef Jan 9 06:44:33.879: INFO: At 2023-01-09 06:39:32 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9a20db9aba088a06f242e4532787ba0da78be3dfac50562851f3b35fa093d4e9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:33.879: INFO: At 2023-01-09 06:39:44 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6fc8a5b8e413db0d962d00e2663f80c84d8d57485c665ccaf14aa15ab156c15a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:33.879: INFO: At 2023-01-09 06:39:55 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a1f237147fd237c54d0f16312d10d297cd00699a6a16f5313cd8a2fb8e3ee358": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:33.879: INFO: At 2023-01-09 06:40:07 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c92a7087f74ebde98066bf7c6a3cdec5f7e6416dd9e2a6060acd82dc2d1cb30f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:33.879: INFO: At 2023-01-09 06:40:20 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8f330a01d8b375c36778be83a1e115960b386ff23e0fdc11c982e6788062d858": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:33.879: INFO: At 2023-01-09 06:40:31 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0713f20f7b193bcbb49efa0ffe41bf82cb556b4bb416cee2473e4275bc901bd2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:33.879: INFO: At 2023-01-09 06:40:46 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "923713530a426058da7d4f3cba97f924e1c7c525aca380b9b322c74c7f0a2a0a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:33.879: INFO: At 2023-01-09 06:40:59 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1ab7212b09b854db246407417398d09edf975dec6c4fe884663567da9eb8efa8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:33.879: INFO: At 2023-01-09 06:41:11 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "881f2fc1220adf63c50fdacd424d98aa0053256823e06a831437f613ea76db64": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:33.879: INFO: At 2023-01-09 06:41:26 +0000 UTC - event for test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "24c9d212906275a0970e1e21e9a617fdc83abc9268e16b7b270bacc46ba80df5": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:34.021: INFO: POD NODE PHASE GRACE CONDITIONS Jan 9 06:44:34.021: INFO: Jan 9 06:44:34.164: INFO: Logging node info for node i-02b09d19b2096261b Jan 9 06:44:34.307: INFO: Node Info: &Node{ObjectMeta:{i-02b09d19b2096261b efa11f02-6d54-4902-bbeb-efecc1f02cd1 29640 0 2023-01-09 06:23:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-02b09d19b2096261b kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.12 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02b09d19b2096261b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-09 06:23:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-09 06:24:22 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-09 06:24:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-09 06:41:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-02b09d19b2096261b,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:24:43 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.12,},NodeAddress{Type:ExternalIP,Address:54.94.119.205,},NodeAddress{Type:InternalDNS,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-119-205.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2da51bd0e40b36975082ac186a369f,SystemUUID:ec2da51b-d0e4-0b36-9750-82ac186a369f,BootID:83891f80-d7aa-4da9-ab10-44706ab3b631,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43453930,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42803458,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967347,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:34.307: INFO: Logging kubelet events for node i-02b09d19b2096261b Jan 9 06:44:34.451: INFO: Logging pods the kubelet thinks is on node i-02b09d19b2096261b Jan 9 06:44:34.599: INFO: dns-controller-69987775c6-mv7h2 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.599: INFO: Container dns-controller ready: true, restart count 0 Jan 9 06:44:34.599: INFO: cilium-operator-5dd44dc49f-fgc7c started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.599: INFO: Container cilium-operator ready: true, restart count 0 Jan 9 06:44:34.599: INFO: etcd-manager-main-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.599: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:44:34.599: INFO: kube-apiserver-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+2 container statuses recorded) Jan 9 06:44:34.599: INFO: Container healthcheck ready: true, restart count 0 Jan 9 06:44:34.599: INFO: Container kube-apiserver ready: true, restart count 1 Jan 9 06:44:34.599: INFO: cilium-97g4h started at 2023-01-09 06:24:28 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:34.599: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:34.599: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:34.599: INFO: aws-cloud-controller-manager-kj4b6 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.599: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 9 06:44:34.599: INFO: kops-controller-8qjsv started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.599: INFO: Container kops-controller ready: true, restart count 0 Jan 9 06:44:34.599: INFO: ebs-csi-node-kblmh started at 2023-01-09 06:24:28 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:34.599: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:34.599: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:34.599: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:34.599: INFO: ebs-csi-controller-89b649c94-zwk75 started at 2023-01-09 06:24:28 +0000 UTC (0+5 container statuses recorded) Jan 9 06:44:34.599: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:44:34.599: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:44:34.599: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:44:34.599: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:34.599: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:34.599: INFO: kube-controller-manager-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.599: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 9 06:44:34.599: INFO: kube-scheduler-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.599: INFO: Container kube-scheduler ready: true, restart count 0 Jan 9 06:44:34.599: INFO: etcd-manager-events-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:34.599: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:44:35.093: INFO: Latency metrics for node i-02b09d19b2096261b Jan 9 06:44:35.093: INFO: Logging node info for node i-0585809501bfc393d Jan 9 06:44:35.237: INFO: Node Info: &Node{ObjectMeta:{i-0585809501bfc393d 8c7c1c77-e673-43a0-81a4-fcf97493bcd8 29271 0 2023-01-09 06:25:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0585809501bfc393d kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-0585809501bfc393d topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.147 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0585809501bfc393d"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-09 06:41:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0585809501bfc393d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.147,},NodeAddress{Type:ExternalIP,Address:54.94.242.160,},NodeAddress{Type:InternalDNS,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-242-160.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec206fc38cf93b7ea12ba838ad958aa3,SystemUUID:ec206fc3-8cf9-3b7e-a12b-a838ad958aa3,BootID:afb66846-6976-49bf-9e61-b54e5af8a3eb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:35.237: INFO: Logging kubelet events for node i-0585809501bfc393d Jan 9 06:44:35.383: INFO: Logging pods the kubelet thinks is on node i-0585809501bfc393d Jan 9 06:44:35.536: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:39:17 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:35.536: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:35.536: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fs9d6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.536: INFO: webserver-7f5969cbc7-bm97p started at 2023-01-09 06:41:26 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:35.536: INFO: alpine-nnp-true-7168054a-3362-4368-bc5e-54ecf20fc5ca started at 2023-01-09 06:39:32 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container alpine-nnp-true-7168054a-3362-4368-bc5e-54ecf20fc5ca ready: false, restart count 0 Jan 9 06:44:35.536: INFO: ebs-csi-node-6djgn started at 2023-01-09 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:35.536: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:35.536: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:35.536: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:35.536: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fm99c started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.536: INFO: execpodjqtvg started at 2023-01-09 06:42:07 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container agnhost-container ready: false, restart count 0 Jan 9 06:44:35.536: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-xtk85 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.536: INFO: alpine-nnp-false-4247b683-138d-4632-a6a3-6d844097c353 started at 2023-01-09 06:41:15 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container alpine-nnp-false-4247b683-138d-4632-a6a3-6d844097c353 ready: false, restart count 0 Jan 9 06:44:35.536: INFO: webserver-66b8c99f6-ls7f4 started at 2023-01-09 06:41:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:35.536: INFO: dns-test-a96d67de-eeb2-4e87-8d66-17c313a4ef5a started at 2023-01-09 06:40:06 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:35.536: INFO: Container jessie-querier ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container querier ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:35.536: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-qm6jm started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.536: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8rq57 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.536: INFO: pod-subpath-test-preprovisionedpv-wp87 started at 2023-01-09 06:39:43 +0000 UTC (2+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Init container init-volume-preprovisionedpv-wp87 ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Init container test-init-subpath-preprovisionedpv-wp87 ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container test-container-subpath-preprovisionedpv-wp87 ready: false, restart count 0 Jan 9 06:44:35.536: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-2jgj6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.536: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-97f28 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.536: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-p6pfg started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.536: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-6hzsb started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.536: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:40:25 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:35.536: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:35.536: INFO: hostexec-i-0585809501bfc393d-v4gr6 started at 2023-01-09 06:39:24 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:35.536: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-cs2m5 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.536: INFO: test-rollover-controller-75rk6 started at 2023-01-09 06:39:32 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:35.536: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-bxrs7 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:35.536: INFO: netserver-0 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:35.536: INFO: cilium-f5j8w started at 2023-01-09 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:35.536: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:35.536: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:35.536: INFO: dns-test-bdcfcfaa-2367-4a35-bb69-c5dd9d18f66c started at 2023-01-09 06:44:29 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:35.536: INFO: Container jessie-querier ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container querier ready: false, restart count 0 Jan 9 06:44:35.536: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:36.066: INFO: Latency metrics for node i-0585809501bfc393d Jan 9 06:44:36.066: INFO: Logging node info for node i-06d5855698aa7cb08 Jan 9 06:44:36.209: INFO: Node Info: &Node{ObjectMeta:{i-06d5855698aa7cb08 bdfd8f9d-5aa1-4cff-806a-849d1c29fcb6 28608 0 2023-01-09 06:25:34 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06d5855698aa7cb08 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-06d5855698aa7cb08 topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.46.163 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-2682":"i-06d5855698aa7cb08","ebs.csi.aws.com":"i-06d5855698aa7cb08"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:39:47 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-09 06:39:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06d5855698aa7cb08,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.46.163,},NodeAddress{Type:ExternalIP,Address:18.228.235.107,},NodeAddress{Type:InternalDNS,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-107.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29e10a0890d67138c4eff5b197ee61,SystemUUID:ec29e10a-0890-d671-38c4-eff5b197ee61,BootID:9352599c-af91-4c75-860f-bb9a2789a998,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-2682^66d24229-8fe8-11ed-abbf-d2969ce11d4b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-2682^66d24229-8fe8-11ed-abbf-d2969ce11d4b,DevicePath:,},},Config:nil,},} Jan 9 06:44:36.209: INFO: Logging kubelet events for node i-06d5855698aa7cb08 Jan 9 06:44:36.354: INFO: Logging pods the kubelet thinks is on node i-06d5855698aa7cb08 Jan 9 06:44:36.504: INFO: hostexec-i-06d5855698aa7cb08-gdgwr started at 2023-01-09 06:39:24 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.504: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:36.504: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-hpb4w started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.504: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.504: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kr2c4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.504: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.504: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-b42jp started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.504: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.504: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pnbg6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.504: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.504: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-mkn6f started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.504: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.504: INFO: webserver-7f5969cbc7-tz85c started at 2023-01-09 06:41:16 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.504: INFO: Container httpd ready: true, restart count 0 Jan 9 06:44:36.504: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-vwlp2 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.504: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.504: INFO: ebs-csi-node-6dzh8 started at 2023-01-09 06:25:35 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:36.504: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:36.504: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:36.504: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:36.504: INFO: netserver-1 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.504: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:36.504: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:39:00 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:36.504: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:44:36.504: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:44:36.504: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:44:36.504: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 9 06:44:36.504: INFO: Container hostpath ready: true, restart count 0 Jan 9 06:44:36.504: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:36.504: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:36.504: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:42:19 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:36.504: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:36.504: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:36.504: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:36.504: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:36.504: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:36.504: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:36.504: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:36.504: INFO: coredns-559769c974-qdhtp started at 2023-01-09 06:26:21 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.504: INFO: Container coredns ready: true, restart count 0 Jan 9 06:44:36.504: INFO: csi-mockplugin-0 started at 2023-01-09 06:40:46 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:36.504: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:36.504: INFO: Container driver-registrar ready: false, restart count 0 Jan 9 06:44:36.504: INFO: Container mock ready: false, restart count 0 Jan 9 06:44:36.504: INFO: cilium-wbn8d started at 2023-01-09 06:25:35 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:36.504: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:36.504: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:36.504: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fncl4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.504: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.504: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8hxw6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.504: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.504: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-wg2gn started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.504: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:36.504: INFO: pod-d7eb61ad-d823-42ce-b057-d1397e0091be started at 2023-01-09 06:39:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:36.504: INFO: Container write-pod ready: false, restart count 0 Jan 9 06:44:37.055: INFO: Latency metrics for node i-06d5855698aa7cb08 Jan 9 06:44:37.055: INFO: Logging node info for node i-087c2326a312024ef Jan 9 06:44:37.197: INFO: Node Info: &Node{ObjectMeta:{i-087c2326a312024ef 45db8cc8-250b-411a-b344-d6bbffe46b7d 29314 0 2023-01-09 06:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-087c2326a312024ef kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-087c2326a312024ef topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.218 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-087c2326a312024ef"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:40:12 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-09 06:41:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-087c2326a312024ef,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.218,},NodeAddress{Type:ExternalIP,Address:18.230.17.62,},NodeAddress{Type:InternalDNS,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-17-62.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec28fbdf9f29233021716214b9f8da3a,SystemUUID:ec28fbdf-9f29-2330-2171-6214b9f8da3a,BootID:4dbf8c75-8514-419f-b443-1a44f89f420f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d49342c84d2a787d],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d49342c84d2a787d,DevicePath:,},},Config:nil,},} Jan 9 06:44:37.198: INFO: Logging kubelet events for node i-087c2326a312024ef Jan 9 06:44:37.342: INFO: Logging pods the kubelet thinks is on node i-087c2326a312024ef Jan 9 06:44:37.505: INFO: sample-webhook-deployment-865554f4d9-sqdbc started at 2023-01-09 06:44:13 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container sample-webhook ready: false, restart count 0 Jan 9 06:44:37.505: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-dlhg8 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:44:37.505: INFO: ebs-csi-node-jvd2b started at 2023-01-09 06:25:35 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:37.505: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:37.505: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:37.505: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:37.505: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-h8zj6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:37.505: INFO: netserver-2 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:37.505: INFO: cilium-gw6mm started at 2023-01-09 06:25:35 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:37.505: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:37.505: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-plbnr started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:37.505: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pvc5d started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:44:37.505: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-r89ft started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:37.505: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8brjv started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:37.505: INFO: hostexec-i-087c2326a312024ef-2vcfq started at 2023-01-09 06:40:20 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:37.505: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pcnbb started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:37.505: INFO: pod-should-be-evicted95f3032f-4412-4cf0-898f-db2e0558e909 started at 2023-01-09 06:44:26 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container bar ready: false, restart count 0 Jan 9 06:44:37.505: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-qxtwd started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:37.505: INFO: pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d started at 2023-01-09 06:39:05 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container write-pod ready: false, restart count 0 Jan 9 06:44:37.505: INFO: test-rs-gj54c started at 2023-01-09 06:44:23 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:37.505: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-b8q64 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:37.505: INFO: pod1 started at 2023-01-09 06:40:38 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:37.505: INFO: oidc-discovery-validator started at 2023-01-09 06:39:17 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container oidc-discovery-validator ready: false, restart count 0 Jan 9 06:44:37.505: INFO: webserver-66b8c99f6-gt295 started at 2023-01-09 06:41:30 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container httpd ready: true, restart count 0 Jan 9 06:44:37.505: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-85kn4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:37.505: INFO: webserver-7f5969cbc7-rf7jj started at 2023-01-09 06:41:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:37.505: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:38.150: INFO: Latency metrics for node i-087c2326a312024ef Jan 9 06:44:38.150: INFO: Logging node info for node i-0c0b18eb32e7da83d Jan 9 06:44:38.292: INFO: Node Info: &Node{ObjectMeta:{i-0c0b18eb32e7da83d f1550edf-43cd-451c-bf01-eb92d6980d45 30343 0 2023-01-09 06:25:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0c0b18eb32e7da83d kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-0c0b18eb32e7da83d topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.55.27 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0c0b18eb32e7da83d"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-09 06:44:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c0b18eb32e7da83d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.55.27,},NodeAddress{Type:ExternalIP,Address:18.228.15.186,},NodeAddress{Type:InternalDNS,Address:i-0c0b18eb32e7da83d.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0c0b18eb32e7da83d.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-15-186.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec267e00a4086801bd60f1191c81f587,SystemUUID:ec267e00-a408-6801-bd60-f1191c81f587,BootID:7c624b34-f575-4c40-8226-aef6612a1a56,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:38.293: INFO: Logging kubelet events for node i-0c0b18eb32e7da83d Jan 9 06:44:38.437: INFO: Logging pods the kubelet thinks is on node i-0c0b18eb32e7da83d Jan 9 06:44:38.586: INFO: ebs-csi-node-m5kg7 started at 2023-01-09 06:25:34 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:38.586: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:38.586: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:38.586: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:38.586: INFO: cilium-vx7vq started at 2023-01-09 06:25:34 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:38.586: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:38.586: INFO: coredns-559769c974-md8gz started at 2023-01-09 06:25:51 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container coredns ready: true, restart count 0 Jan 9 06:44:38.586: INFO: coredns-autoscaler-7cb5c5b969-4vwwx started at 2023-01-09 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container autoscaler ready: true, restart count 0 Jan 9 06:44:38.586: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-sf7zm started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.586: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-mwm9d started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.586: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-twg96 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.586: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-ft8kr started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.586: INFO: hostpath-client started at 2023-01-09 06:40:50 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container hostpath-client ready: false, restart count 0 Jan 9 06:44:38.586: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kpwrs started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:44:38.586: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kxqfw started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.586: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-ck9r4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.586: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-lknfk started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.586: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-cfps7 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.586: INFO: webserver-pod started at 2023-01-09 06:39:16 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container agnhost ready: false, restart count 0 Jan 9 06:44:38.586: INFO: netserver-3 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:38.586: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-njsp9 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:38.586: INFO: webserver-66b8c99f6-b9md9 started at 2023-01-09 06:44:35 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:38.586: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:39.134: INFO: Latency metrics for node i-0c0b18eb32e7da83d [DeferCleanup (Each)] [sig-auth] ServiceAccounts tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "svcaccounts-1924" for this suite. �[38;5;243m01/09/23 06:44:39.134�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sclient\sUpdate\sDemo\sshould\sscale\sa\sreplication\scontroller\s\s\[Conformance\]$'
test/e2e/kubectl/kubectl.go:2431 k8s.io/kubernetes/test/e2e/kubectl.validateController({0x801e128, 0xc002260000}, {0xc000de4450?, 0xc00010e480?}, 0x2, {0x75cf5a1, 0xb}, {0x75e78bb, 0x10}, 0xc00360c2a0, ...) test/e2e/kubectl/kubectl.go:2431 +0x49d k8s.io/kubernetes/test/e2e/kubectl.glob..func1.6.3() test/e2e/kubectl/kubectl.go:357 +0x205from junit_01.xml
[BeforeEach] [sig-cli] Kubectl client set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/09/23 06:39:16.008�[0m Jan 9 06:39:16.008: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename kubectl �[38;5;243m01/09/23 06:39:16.009�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/09/23 06:39:16.435�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/09/23 06:39:16.717�[0m [BeforeEach] [sig-cli] Kubectl client test/e2e/framework/metrics/init/init.go:31 [BeforeEach] [sig-cli] Kubectl client test/e2e/kubectl/kubectl.go:274 [BeforeEach] Update Demo test/e2e/kubectl/kubectl.go:326 [It] should scale a replication controller [Conformance] test/e2e/kubectl/kubectl.go:352 �[1mSTEP:�[0m creating a replication controller �[38;5;243m01/09/23 06:39:17�[0m Jan 9 06:39:17.000: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 create -f -' Jan 9 06:39:17.794: INFO: stderr: "" Jan 9 06:39:17.794: INFO: stdout: "replicationcontroller/update-demo-nautilus created\n" �[1mSTEP:�[0m waiting for all containers in name=update-demo pods to come up. �[38;5;243m01/09/23 06:39:17.794�[0m Jan 9 06:39:17.794: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:39:18.453: INFO: stderr: "" Jan 9 06:39:18.453: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:39:18.453: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:39:18.971: INFO: stderr: "" Jan 9 06:39:18.971: INFO: stdout: "" Jan 9 06:39:18.971: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:39:23.972: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:39:25.204: INFO: stderr: "" Jan 9 06:39:25.204: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:39:25.204: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:39:25.716: INFO: stderr: "" Jan 9 06:39:25.716: INFO: stdout: "" Jan 9 06:39:25.716: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:39:30.716: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:39:31.370: INFO: stderr: "" Jan 9 06:39:31.371: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:39:31.371: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:39:31.903: INFO: stderr: "" Jan 9 06:39:31.903: INFO: stdout: "" Jan 9 06:39:31.903: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:39:36.904: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:39:37.580: INFO: stderr: "" Jan 9 06:39:37.580: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:39:37.580: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:39:38.100: INFO: stderr: "" Jan 9 06:39:38.100: INFO: stdout: "" Jan 9 06:39:38.100: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:39:43.100: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:39:43.764: INFO: stderr: "" Jan 9 06:39:43.764: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:39:43.764: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:39:44.274: INFO: stderr: "" Jan 9 06:39:44.274: INFO: stdout: "" Jan 9 06:39:44.274: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:39:49.274: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:39:49.935: INFO: stderr: "" Jan 9 06:39:49.935: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:39:49.935: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:39:50.464: INFO: stderr: "" Jan 9 06:39:50.464: INFO: stdout: "" Jan 9 06:39:50.464: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:39:55.464: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:39:56.150: INFO: stderr: "" Jan 9 06:39:56.150: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:39:56.150: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:39:56.676: INFO: stderr: "" Jan 9 06:39:56.676: INFO: stdout: "" Jan 9 06:39:56.676: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:40:01.677: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:40:02.368: INFO: stderr: "" Jan 9 06:40:02.368: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:40:02.368: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:40:02.893: INFO: stderr: "" Jan 9 06:40:02.893: INFO: stdout: "" Jan 9 06:40:02.893: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:40:07.893: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:40:08.567: INFO: stderr: "" Jan 9 06:40:08.567: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:40:08.567: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:40:09.096: INFO: stderr: "" Jan 9 06:40:09.096: INFO: stdout: "" Jan 9 06:40:09.096: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:40:14.096: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:40:14.764: INFO: stderr: "" Jan 9 06:40:14.764: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:40:14.764: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:40:15.309: INFO: stderr: "" Jan 9 06:40:15.309: INFO: stdout: "" Jan 9 06:40:15.309: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:40:20.310: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:40:20.972: INFO: stderr: "" Jan 9 06:40:20.972: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:40:20.972: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:40:21.489: INFO: stderr: "" Jan 9 06:40:21.489: INFO: stdout: "" Jan 9 06:40:21.489: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:40:26.489: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:40:27.149: INFO: stderr: "" Jan 9 06:40:27.149: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:40:27.149: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:40:27.683: INFO: stderr: "" Jan 9 06:40:27.683: INFO: stdout: "" Jan 9 06:40:27.683: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:40:32.683: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:40:33.337: INFO: stderr: "" Jan 9 06:40:33.337: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:40:33.337: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:40:33.851: INFO: stderr: "" Jan 9 06:40:33.851: INFO: stdout: "" Jan 9 06:40:33.851: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:40:38.851: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:40:39.514: INFO: stderr: "" Jan 9 06:40:39.514: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:40:39.514: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:40:40.030: INFO: stderr: "" Jan 9 06:40:40.030: INFO: stdout: "" Jan 9 06:40:40.030: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:40:45.031: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:40:45.718: INFO: stderr: "" Jan 9 06:40:45.718: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:40:45.718: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:40:46.234: INFO: stderr: "" Jan 9 06:40:46.234: INFO: stdout: "" Jan 9 06:40:46.234: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:40:51.235: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:40:51.896: INFO: stderr: "" Jan 9 06:40:51.896: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:40:51.897: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:40:52.415: INFO: stderr: "" Jan 9 06:40:52.415: INFO: stdout: "" Jan 9 06:40:52.415: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:40:57.416: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:40:58.072: INFO: stderr: "" Jan 9 06:40:58.072: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:40:58.072: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:40:58.586: INFO: stderr: "" Jan 9 06:40:58.586: INFO: stdout: "" Jan 9 06:40:58.586: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:41:03.586: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:41:04.246: INFO: stderr: "" Jan 9 06:41:04.246: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:41:04.246: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:41:04.785: INFO: stderr: "" Jan 9 06:41:04.785: INFO: stdout: "" Jan 9 06:41:04.785: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:41:09.785: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:41:10.440: INFO: stderr: "" Jan 9 06:41:10.440: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:41:10.440: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:41:10.957: INFO: stderr: "" Jan 9 06:41:10.957: INFO: stdout: "" Jan 9 06:41:10.957: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:41:15.957: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:41:16.618: INFO: stderr: "" Jan 9 06:41:16.618: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:41:16.618: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:41:17.147: INFO: stderr: "" Jan 9 06:41:17.147: INFO: stdout: "" Jan 9 06:41:17.147: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:41:22.147: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:41:22.827: INFO: stderr: "" Jan 9 06:41:22.827: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:41:22.827: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:41:23.359: INFO: stderr: "" Jan 9 06:41:23.359: INFO: stdout: "" Jan 9 06:41:23.359: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:41:28.360: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:41:29.023: INFO: stderr: "" Jan 9 06:41:29.023: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:41:29.023: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:41:29.539: INFO: stderr: "" Jan 9 06:41:29.539: INFO: stdout: "" Jan 9 06:41:29.539: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:41:34.539: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:41:35.216: INFO: stderr: "" Jan 9 06:41:35.216: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:41:35.216: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:41:35.727: INFO: stderr: "" Jan 9 06:41:35.727: INFO: stdout: "" Jan 9 06:41:35.727: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:41:40.728: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:41:41.389: INFO: stderr: "" Jan 9 06:41:41.389: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:41:41.389: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:41:41.913: INFO: stderr: "" Jan 9 06:41:41.913: INFO: stdout: "" Jan 9 06:41:41.913: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:41:46.914: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:41:47.577: INFO: stderr: "" Jan 9 06:41:47.577: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:41:47.577: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:41:48.107: INFO: stderr: "" Jan 9 06:41:48.107: INFO: stdout: "" Jan 9 06:41:48.107: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:41:53.108: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:41:53.767: INFO: stderr: "" Jan 9 06:41:53.767: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:41:53.767: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:41:54.286: INFO: stderr: "" Jan 9 06:41:54.286: INFO: stdout: "" Jan 9 06:41:54.286: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:41:59.287: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:41:59.944: INFO: stderr: "" Jan 9 06:41:59.944: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:41:59.944: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:42:00.472: INFO: stderr: "" Jan 9 06:42:00.472: INFO: stdout: "" Jan 9 06:42:00.472: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:42:05.473: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:42:06.143: INFO: stderr: "" Jan 9 06:42:06.143: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:42:06.143: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:42:06.659: INFO: stderr: "" Jan 9 06:42:06.659: INFO: stdout: "" Jan 9 06:42:06.659: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:42:11.659: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:42:12.313: INFO: stderr: "" Jan 9 06:42:12.313: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:42:12.313: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:42:12.834: INFO: stderr: "" Jan 9 06:42:12.834: INFO: stdout: "" Jan 9 06:42:12.834: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:42:17.834: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:42:18.512: INFO: stderr: "" Jan 9 06:42:18.512: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:42:18.512: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:42:19.053: INFO: stderr: "" Jan 9 06:42:19.053: INFO: stdout: "" Jan 9 06:42:19.053: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:42:24.054: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:42:24.716: INFO: stderr: "" Jan 9 06:42:24.716: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:42:24.716: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:42:25.231: INFO: stderr: "" Jan 9 06:42:25.231: INFO: stdout: "" Jan 9 06:42:25.231: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:42:30.231: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:42:30.901: INFO: stderr: "" Jan 9 06:42:30.901: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:42:30.901: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:42:31.453: INFO: stderr: "" Jan 9 06:42:31.453: INFO: stdout: "" Jan 9 06:42:31.453: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:42:36.454: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:42:37.116: INFO: stderr: "" Jan 9 06:42:37.116: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:42:37.116: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:42:37.632: INFO: stderr: "" Jan 9 06:42:37.632: INFO: stdout: "" Jan 9 06:42:37.632: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:42:42.633: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:42:43.313: INFO: stderr: "" Jan 9 06:42:43.313: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:42:43.313: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:42:43.851: INFO: stderr: "" Jan 9 06:42:43.851: INFO: stdout: "" Jan 9 06:42:43.851: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:42:48.851: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:42:49.529: INFO: stderr: "" Jan 9 06:42:49.529: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:42:49.529: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:42:50.046: INFO: stderr: "" Jan 9 06:42:50.046: INFO: stdout: "" Jan 9 06:42:50.046: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:42:55.046: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:42:55.739: INFO: stderr: "" Jan 9 06:42:55.739: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:42:55.739: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:42:56.269: INFO: stderr: "" Jan 9 06:42:56.269: INFO: stdout: "" Jan 9 06:42:56.269: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:43:01.269: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:43:01.935: INFO: stderr: "" Jan 9 06:43:01.935: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:43:01.935: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:43:02.471: INFO: stderr: "" Jan 9 06:43:02.471: INFO: stdout: "" Jan 9 06:43:02.471: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:43:07.472: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:43:08.164: INFO: stderr: "" Jan 9 06:43:08.164: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:43:08.164: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:43:08.683: INFO: stderr: "" Jan 9 06:43:08.683: INFO: stdout: "" Jan 9 06:43:08.683: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:43:13.684: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:43:14.344: INFO: stderr: "" Jan 9 06:43:14.344: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:43:14.344: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:43:14.863: INFO: stderr: "" Jan 9 06:43:14.863: INFO: stdout: "" Jan 9 06:43:14.863: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:43:19.863: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:43:20.544: INFO: stderr: "" Jan 9 06:43:20.544: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:43:20.544: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:43:21.057: INFO: stderr: "" Jan 9 06:43:21.057: INFO: stdout: "" Jan 9 06:43:21.057: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:43:26.058: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:43:26.720: INFO: stderr: "" Jan 9 06:43:26.720: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:43:26.720: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:43:27.251: INFO: stderr: "" Jan 9 06:43:27.251: INFO: stdout: "" Jan 9 06:43:27.251: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:43:32.251: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:43:32.931: INFO: stderr: "" Jan 9 06:43:32.931: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:43:32.931: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:43:33.443: INFO: stderr: "" Jan 9 06:43:33.443: INFO: stdout: "" Jan 9 06:43:33.443: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:43:38.444: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:43:39.122: INFO: stderr: "" Jan 9 06:43:39.122: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:43:39.122: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:43:39.651: INFO: stderr: "" Jan 9 06:43:39.651: INFO: stdout: "" Jan 9 06:43:39.651: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:43:44.652: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:43:45.331: INFO: stderr: "" Jan 9 06:43:45.331: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:43:45.332: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:43:45.855: INFO: stderr: "" Jan 9 06:43:45.855: INFO: stdout: "" Jan 9 06:43:45.855: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:43:50.855: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:43:51.521: INFO: stderr: "" Jan 9 06:43:51.521: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:43:51.521: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:43:52.043: INFO: stderr: "" Jan 9 06:43:52.043: INFO: stdout: "" Jan 9 06:43:52.043: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:43:57.044: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:43:57.710: INFO: stderr: "" Jan 9 06:43:57.710: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:43:57.710: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:43:58.254: INFO: stderr: "" Jan 9 06:43:58.254: INFO: stdout: "" Jan 9 06:43:58.254: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:44:03.255: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:44:03.925: INFO: stderr: "" Jan 9 06:44:03.925: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:44:03.925: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:44:04.457: INFO: stderr: "" Jan 9 06:44:04.457: INFO: stdout: "" Jan 9 06:44:04.457: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:44:09.457: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:44:10.147: INFO: stderr: "" Jan 9 06:44:10.147: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:44:10.147: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:44:10.666: INFO: stderr: "" Jan 9 06:44:10.666: INFO: stdout: "" Jan 9 06:44:10.666: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:44:15.666: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo' Jan 9 06:44:16.329: INFO: stderr: "" Jan 9 06:44:16.329: INFO: stdout: "update-demo-nautilus-gqbgn update-demo-nautilus-l62pj " Jan 9 06:44:16.329: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods update-demo-nautilus-gqbgn -o template --template={{if (exists . "status" "containerStatuses")}}{{range .status.containerStatuses}}{{if (and (eq .name "update-demo") (exists . "state" "running"))}}true{{end}}{{end}}{{end}}' Jan 9 06:44:16.863: INFO: stderr: "" Jan 9 06:44:16.863: INFO: stdout: "" Jan 9 06:44:16.863: INFO: update-demo-nautilus-gqbgn is created but not running Jan 9 06:44:21.864: FAIL: Timed out after 300 seconds waiting for name=update-demo pods to reach valid state Full Stack Trace k8s.io/kubernetes/test/e2e/kubectl.validateController({0x801e128, 0xc002260000}, {0xc000de4450?, 0xc00010e480?}, 0x2, {0x75cf5a1, 0xb}, {0x75e78bb, 0x10}, 0xc00360c2a0, ...) test/e2e/kubectl/kubectl.go:2431 +0x49d k8s.io/kubernetes/test/e2e/kubectl.glob..func1.6.3() test/e2e/kubectl/kubectl.go:357 +0x205 �[1mSTEP:�[0m using delete to clean up resources �[38;5;243m01/09/23 06:44:21.864�[0m Jan 9 06:44:21.864: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 delete --grace-period=0 --force -f -' Jan 9 06:44:22.558: INFO: stderr: "Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.\n" Jan 9 06:44:22.558: INFO: stdout: "replicationcontroller \"update-demo-nautilus\" force deleted\n" Jan 9 06:44:22.558: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get rc,svc -l name=update-demo --no-headers' Jan 9 06:44:23.241: INFO: stderr: "No resources found in kubectl-145 namespace.\n" Jan 9 06:44:23.241: INFO: stdout: "" Jan 9 06:44:23.241: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/8b946944-8fe5-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-145 get pods -l name=update-demo -o go-template={{ range .items }}{{ if not .metadata.deletionTimestamp }}{{ .metadata.name }}{{ "\n" }}{{ end }}{{ end }}' Jan 9 06:44:23.769: INFO: stderr: "" Jan 9 06:44:23.769: INFO: stdout: "" [AfterEach] [sig-cli] Kubectl client test/e2e/framework/node/init/init.go:32 Jan 9 06:44:23.770: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-cli] Kubectl client test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-cli] Kubectl client dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/09/23 06:44:23.913�[0m �[1mSTEP:�[0m Collecting events from namespace "kubectl-145". �[38;5;243m01/09/23 06:44:23.913�[0m �[1mSTEP:�[0m Found 25 events. �[38;5;243m01/09/23 06:44:24.059�[0m Jan 9 06:44:24.060: INFO: At 2023-01-09 06:39:17 +0000 UTC - event for update-demo-nautilus: {replication-controller } SuccessfulCreate: Created pod: update-demo-nautilus-l62pj Jan 9 06:44:24.060: INFO: At 2023-01-09 06:39:17 +0000 UTC - event for update-demo-nautilus: {replication-controller } SuccessfulCreate: Created pod: update-demo-nautilus-gqbgn Jan 9 06:44:24.060: INFO: At 2023-01-09 06:39:17 +0000 UTC - event for update-demo-nautilus-gqbgn: {default-scheduler } Scheduled: Successfully assigned kubectl-145/update-demo-nautilus-gqbgn to i-0585809501bfc393d Jan 9 06:44:24.060: INFO: At 2023-01-09 06:39:17 +0000 UTC - event for update-demo-nautilus-l62pj: {default-scheduler } Scheduled: Successfully assigned kubectl-145/update-demo-nautilus-l62pj to i-087c2326a312024ef Jan 9 06:44:24.060: INFO: At 2023-01-09 06:39:18 +0000 UTC - event for update-demo-nautilus-gqbgn: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "07e0946fc800b37f366770037bacea168dc6467f97e966bace3d414bb12bcec4": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:39:18 +0000 UTC - event for update-demo-nautilus-l62pj: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "17a55239971b2d4f45c53fcd28cbac0f39bb462ff798a8d91df20d3fc459c6b2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:39:29 +0000 UTC - event for update-demo-nautilus-l62pj: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "74786836083df1897fb2c62879b5c90b5bf8c539ada733c6c402e6b41d55892a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:39:30 +0000 UTC - event for update-demo-nautilus-gqbgn: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a0d1b789d3019d2fd95fa8d73b4c8a10dd77d79775af802a8604c48b367cbf12": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:39:40 +0000 UTC - event for update-demo-nautilus-l62pj: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "66484f664f318b31c9400edb4f3eb09fac573ce976a1bf86aad47812232b36db": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:39:43 +0000 UTC - event for update-demo-nautilus-gqbgn: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "71ec10d668d7e9f9a0ed958a79cbf36b91d6c9d40c2a4e562d0b3ff678d8a713": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:39:53 +0000 UTC - event for update-demo-nautilus-l62pj: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3030758658c3a820b04237676baea0012063121c2e999f92e28b50af4aa9749b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:39:56 +0000 UTC - event for update-demo-nautilus-gqbgn: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "493ab37b90038af5fe488da7c492303a46cf84c1fb979b643a8088f05c081950": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:40:07 +0000 UTC - event for update-demo-nautilus-l62pj: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "185dd8ac0d37b77041ea3bd7849c2ae1bd9c99998c63e7ee61f96eb4aa036ea1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:40:09 +0000 UTC - event for update-demo-nautilus-gqbgn: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "eb1b7e36cce4826c44232086e3764ccea6a4765cfbb1033cc7626631076f2f80": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:40:21 +0000 UTC - event for update-demo-nautilus-gqbgn: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6c9ceecea7e8fd106693c1b2e10cc03b7aab8439e2912f5dd31cf913c166f287": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:40:21 +0000 UTC - event for update-demo-nautilus-l62pj: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ecf80a94b874dd3743257fff655a26d5153f8b7bcb85e435934856d6f7e8b406": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:40:35 +0000 UTC - event for update-demo-nautilus-gqbgn: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a2b33d8145f66a809c37516ab8a95299bc72d013f045104c4c00f20fef46e08a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:40:36 +0000 UTC - event for update-demo-nautilus-l62pj: {kubelet i-087c2326a312024ef} Pulling: Pulling image "registry.k8s.io/e2e-test-images/nautilus:1.7" Jan 9 06:44:24.060: INFO: At 2023-01-09 06:40:42 +0000 UTC - event for update-demo-nautilus-l62pj: {kubelet i-087c2326a312024ef} Pulled: Successfully pulled image "registry.k8s.io/e2e-test-images/nautilus:1.7" in 5.363489994s (5.36349712s including waiting) Jan 9 06:44:24.060: INFO: At 2023-01-09 06:40:42 +0000 UTC - event for update-demo-nautilus-l62pj: {kubelet i-087c2326a312024ef} Created: Created container update-demo Jan 9 06:44:24.060: INFO: At 2023-01-09 06:40:42 +0000 UTC - event for update-demo-nautilus-l62pj: {kubelet i-087c2326a312024ef} Started: Started container update-demo Jan 9 06:44:24.060: INFO: At 2023-01-09 06:40:50 +0000 UTC - event for update-demo-nautilus-gqbgn: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3408a11479ec0e619b7802c05c4af44dd5f363a7409f3f0a91806645bceadc91": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:41:04 +0000 UTC - event for update-demo-nautilus-gqbgn: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8ca08d805fb64679e38a71114ce461950c6610e1bd6fb19096d3e5b482371b05": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:41:16 +0000 UTC - event for update-demo-nautilus-gqbgn: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f9b743270cb21f90312a73b0c6a2dfa3e8aff3e8ee61fa8f3ed2b0a4cb2fa7fb": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:24.060: INFO: At 2023-01-09 06:44:22 +0000 UTC - event for update-demo-nautilus-l62pj: {kubelet i-087c2326a312024ef} Killing: Stopping container update-demo Jan 9 06:44:24.201: INFO: POD NODE PHASE GRACE CONDITIONS Jan 9 06:44:24.201: INFO: Jan 9 06:44:24.344: INFO: Logging node info for node i-02b09d19b2096261b Jan 9 06:44:24.486: INFO: Node Info: &Node{ObjectMeta:{i-02b09d19b2096261b efa11f02-6d54-4902-bbeb-efecc1f02cd1 29640 0 2023-01-09 06:23:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-02b09d19b2096261b kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.12 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02b09d19b2096261b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-09 06:23:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-09 06:24:22 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-09 06:24:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-09 06:41:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-02b09d19b2096261b,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:24:43 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.12,},NodeAddress{Type:ExternalIP,Address:54.94.119.205,},NodeAddress{Type:InternalDNS,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-119-205.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2da51bd0e40b36975082ac186a369f,SystemUUID:ec2da51b-d0e4-0b36-9750-82ac186a369f,BootID:83891f80-d7aa-4da9-ab10-44706ab3b631,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43453930,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42803458,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967347,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:24.486: INFO: Logging kubelet events for node i-02b09d19b2096261b Jan 9 06:44:24.631: INFO: Logging pods the kubelet thinks is on node i-02b09d19b2096261b Jan 9 06:44:24.780: INFO: aws-cloud-controller-manager-kj4b6 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.780: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 9 06:44:24.780: INFO: kops-controller-8qjsv started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.780: INFO: Container kops-controller ready: true, restart count 0 Jan 9 06:44:24.780: INFO: ebs-csi-node-kblmh started at 2023-01-09 06:24:28 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:24.780: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:24.780: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:24.780: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:24.780: INFO: ebs-csi-controller-89b649c94-zwk75 started at 2023-01-09 06:24:28 +0000 UTC (0+5 container statuses recorded) Jan 9 06:44:24.780: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:44:24.780: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:44:24.780: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:44:24.780: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:24.780: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:24.780: INFO: kube-controller-manager-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.780: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 9 06:44:24.780: INFO: kube-scheduler-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.780: INFO: Container kube-scheduler ready: true, restart count 0 Jan 9 06:44:24.780: INFO: etcd-manager-events-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.780: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:44:24.780: INFO: dns-controller-69987775c6-mv7h2 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.780: INFO: Container dns-controller ready: true, restart count 0 Jan 9 06:44:24.780: INFO: cilium-operator-5dd44dc49f-fgc7c started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.780: INFO: Container cilium-operator ready: true, restart count 0 Jan 9 06:44:24.780: INFO: etcd-manager-main-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.780: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:44:24.780: INFO: kube-apiserver-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+2 container statuses recorded) Jan 9 06:44:24.780: INFO: Container healthcheck ready: true, restart count 0 Jan 9 06:44:24.780: INFO: Container kube-apiserver ready: true, restart count 1 Jan 9 06:44:24.780: INFO: cilium-97g4h started at 2023-01-09 06:24:28 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:24.780: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:24.780: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:25.261: INFO: Latency metrics for node i-02b09d19b2096261b Jan 9 06:44:25.261: INFO: Logging node info for node i-0585809501bfc393d Jan 9 06:44:25.403: INFO: Node Info: &Node{ObjectMeta:{i-0585809501bfc393d 8c7c1c77-e673-43a0-81a4-fcf97493bcd8 29271 0 2023-01-09 06:25:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0585809501bfc393d kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-0585809501bfc393d topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.147 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0585809501bfc393d"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-09 06:41:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0585809501bfc393d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.147,},NodeAddress{Type:ExternalIP,Address:54.94.242.160,},NodeAddress{Type:InternalDNS,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-242-160.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec206fc38cf93b7ea12ba838ad958aa3,SystemUUID:ec206fc3-8cf9-3b7e-a12b-a838ad958aa3,BootID:afb66846-6976-49bf-9e61-b54e5af8a3eb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:25.403: INFO: Logging kubelet events for node i-0585809501bfc393d Jan 9 06:44:25.547: INFO: Logging pods the kubelet thinks is on node i-0585809501bfc393d Jan 9 06:44:25.697: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fs9d6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:25.697: INFO: webserver-7f5969cbc7-bm97p started at 2023-01-09 06:41:26 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:25.697: INFO: alpine-nnp-true-7168054a-3362-4368-bc5e-54ecf20fc5ca started at 2023-01-09 06:39:32 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container alpine-nnp-true-7168054a-3362-4368-bc5e-54ecf20fc5ca ready: false, restart count 0 Jan 9 06:44:25.697: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:39:17 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:25.697: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:25.697: INFO: ebs-csi-node-6djgn started at 2023-01-09 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:25.697: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:25.697: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:25.697: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:25.697: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fm99c started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:25.697: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-xtk85 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:25.697: INFO: alpine-nnp-false-4247b683-138d-4632-a6a3-6d844097c353 started at 2023-01-09 06:41:15 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container alpine-nnp-false-4247b683-138d-4632-a6a3-6d844097c353 ready: false, restart count 0 Jan 9 06:44:25.697: INFO: webserver-66b8c99f6-ls7f4 started at 2023-01-09 06:41:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:25.697: INFO: execpodjqtvg started at 2023-01-09 06:42:07 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container agnhost-container ready: false, restart count 0 Jan 9 06:44:25.697: INFO: dns-test-a96d67de-eeb2-4e87-8d66-17c313a4ef5a started at 2023-01-09 06:40:06 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:25.697: INFO: Container jessie-querier ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Container querier ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:25.697: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-qm6jm started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:25.697: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8rq57 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:25.697: INFO: pod-subpath-test-preprovisionedpv-wp87 started at 2023-01-09 06:39:43 +0000 UTC (2+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Init container init-volume-preprovisionedpv-wp87 ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Init container test-init-subpath-preprovisionedpv-wp87 ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Container test-container-subpath-preprovisionedpv-wp87 ready: false, restart count 0 Jan 9 06:44:25.697: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-2jgj6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:25.697: INFO: hostexec-i-0585809501bfc393d-v4gr6 started at 2023-01-09 06:39:24 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:25.697: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-cs2m5 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:25.697: INFO: test-rollover-controller-75rk6 started at 2023-01-09 06:39:32 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:25.697: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-97f28 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:25.697: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-p6pfg started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:25.697: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-6hzsb started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:25.697: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:40:25 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:25.697: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:25.697: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:25.697: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-bxrs7 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:25.697: INFO: netserver-0 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:25.697: INFO: cilium-f5j8w started at 2023-01-09 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:25.697: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:25.697: INFO: netserver-0 started at 2023-01-09 06:39:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:25.697: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:26.215: INFO: Latency metrics for node i-0585809501bfc393d Jan 9 06:44:26.215: INFO: Logging node info for node i-06d5855698aa7cb08 Jan 9 06:44:26.357: INFO: Node Info: &Node{ObjectMeta:{i-06d5855698aa7cb08 bdfd8f9d-5aa1-4cff-806a-849d1c29fcb6 28608 0 2023-01-09 06:25:34 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06d5855698aa7cb08 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-06d5855698aa7cb08 topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.46.163 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-2682":"i-06d5855698aa7cb08","ebs.csi.aws.com":"i-06d5855698aa7cb08"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:39:47 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-09 06:39:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06d5855698aa7cb08,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.46.163,},NodeAddress{Type:ExternalIP,Address:18.228.235.107,},NodeAddress{Type:InternalDNS,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-107.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29e10a0890d67138c4eff5b197ee61,SystemUUID:ec29e10a-0890-d671-38c4-eff5b197ee61,BootID:9352599c-af91-4c75-860f-bb9a2789a998,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-2682^66d24229-8fe8-11ed-abbf-d2969ce11d4b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-2682^66d24229-8fe8-11ed-abbf-d2969ce11d4b,DevicePath:,},},Config:nil,},} Jan 9 06:44:26.357: INFO: Logging kubelet events for node i-06d5855698aa7cb08 Jan 9 06:44:26.501: INFO: Logging pods the kubelet thinks is on node i-06d5855698aa7cb08 Jan 9 06:44:26.652: INFO: webserver-7f5969cbc7-tz85c started at 2023-01-09 06:41:16 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:26.652: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-vwlp2 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:26.652: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pnbg6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:26.652: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-mkn6f started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:26.652: INFO: pod-fe7ddf40-2506-41f1-af53-09d0de5705e7 started at 2023-01-09 06:39:29 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container write-pod ready: false, restart count 0 Jan 9 06:44:26.652: INFO: netserver-1 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:26.652: INFO: ebs-csi-node-6dzh8 started at 2023-01-09 06:25:35 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:26.652: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:26.652: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:26.652: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:26.652: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:42:19 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:26.652: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:26.652: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:26.652: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:26.652: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:26.652: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:26.652: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:26.652: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:26.652: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:39:00 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:26.652: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:44:26.652: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:44:26.652: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:44:26.652: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 9 06:44:26.652: INFO: Container hostpath ready: true, restart count 0 Jan 9 06:44:26.652: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:26.652: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:26.652: INFO: netserver-1 started at 2023-01-09 06:39:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:26.652: INFO: csi-mockplugin-0 started at 2023-01-09 06:40:46 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:26.652: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:26.652: INFO: Container driver-registrar ready: false, restart count 0 Jan 9 06:44:26.652: INFO: Container mock ready: false, restart count 0 Jan 9 06:44:26.652: INFO: coredns-559769c974-qdhtp started at 2023-01-09 06:26:21 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container coredns ready: true, restart count 0 Jan 9 06:44:26.652: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fncl4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:26.652: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8hxw6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:26.652: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-wg2gn started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:26.652: INFO: pod-d7eb61ad-d823-42ce-b057-d1397e0091be started at 2023-01-09 06:39:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container write-pod ready: false, restart count 0 Jan 9 06:44:26.652: INFO: cilium-wbn8d started at 2023-01-09 06:25:35 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:26.652: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:26.652: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-hpb4w started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:26.652: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kr2c4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:26.652: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-b42jp started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:26.652: INFO: hostexec-i-06d5855698aa7cb08-gdgwr started at 2023-01-09 06:39:24 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:26.652: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:27.192: INFO: Latency metrics for node i-06d5855698aa7cb08 Jan 9 06:44:27.192: INFO: Logging node info for node i-087c2326a312024ef Jan 9 06:44:27.334: INFO: Node Info: &Node{ObjectMeta:{i-087c2326a312024ef 45db8cc8-250b-411a-b344-d6bbffe46b7d 29314 0 2023-01-09 06:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-087c2326a312024ef kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-087c2326a312024ef topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.218 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-087c2326a312024ef"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:40:12 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-09 06:41:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-087c2326a312024ef,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.218,},NodeAddress{Type:ExternalIP,Address:18.230.17.62,},NodeAddress{Type:InternalDNS,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-17-62.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec28fbdf9f29233021716214b9f8da3a,SystemUUID:ec28fbdf-9f29-2330-2171-6214b9f8da3a,BootID:4dbf8c75-8514-419f-b443-1a44f89f420f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d49342c84d2a787d],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d49342c84d2a787d,DevicePath:,},},Config:nil,},} Jan 9 06:44:27.335: INFO: Logging kubelet events for node i-087c2326a312024ef Jan 9 06:44:27.479: INFO: Logging pods the kubelet thinks is on node i-087c2326a312024ef Jan 9 06:44:27.628: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-r89ft started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:27.628: INFO: cilium-gw6mm started at 2023-01-09 06:25:35 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:27.628: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:27.628: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-plbnr started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:27.628: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pvc5d started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:44:27.628: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8brjv started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:27.628: INFO: hostexec-i-087c2326a312024ef-2vcfq started at 2023-01-09 06:40:20 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:27.628: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pcnbb started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:27.628: INFO: pod-should-be-evicted95f3032f-4412-4cf0-898f-db2e0558e909 started at 2023-01-09 06:44:26 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container bar ready: false, restart count 0 Jan 9 06:44:27.628: INFO: netserver-2 started at 2023-01-09 06:39:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:27.628: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-qxtwd started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:27.628: INFO: pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d started at 2023-01-09 06:39:05 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container write-pod ready: false, restart count 0 Jan 9 06:44:27.628: INFO: test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db started at 2023-01-09 06:39:32 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container agnhost-container ready: false, restart count 0 Jan 9 06:44:27.628: INFO: test-rs-gj54c started at 2023-01-09 06:44:23 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:27.628: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-b8q64 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:27.628: INFO: webserver-7f5969cbc7-rf7jj started at 2023-01-09 06:41:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:27.628: INFO: pod1 started at 2023-01-09 06:40:38 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:27.628: INFO: oidc-discovery-validator started at 2023-01-09 06:39:17 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container oidc-discovery-validator ready: false, restart count 0 Jan 9 06:44:27.628: INFO: webserver-66b8c99f6-gt295 started at 2023-01-09 06:41:30 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:27.628: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-85kn4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:27.628: INFO: local-client started at 2023-01-09 06:42:01 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container local-client ready: true, restart count 0 Jan 9 06:44:27.628: INFO: sample-webhook-deployment-865554f4d9-sqdbc started at 2023-01-09 06:44:13 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container sample-webhook ready: false, restart count 0 Jan 9 06:44:27.628: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-dlhg8 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:44:27.628: INFO: ebs-csi-node-jvd2b started at 2023-01-09 06:25:35 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:27.628: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:27.628: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:27.628: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:27.628: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-h8zj6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:27.628: INFO: netserver-2 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:27.628: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:28.155: INFO: Latency metrics for node i-087c2326a312024ef Jan 9 06:44:28.155: INFO: Logging node info for node i-0c0b18eb32e7da83d Jan 9 06:44:28.297: INFO: Node Info: &Node{ObjectMeta:{i-0c0b18eb32e7da83d f1550edf-43cd-451c-bf01-eb92d6980d45 30343 0 2023-01-09 06:25:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0c0b18eb32e7da83d kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-0c0b18eb32e7da83d topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.55.27 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0c0b18eb32e7da83d"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-09 06:44:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c0b18eb32e7da83d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.55.27,},NodeAddress{Type:ExternalIP,Address:18.228.15.186,},NodeAddress{Type:InternalDNS,Address:i-0c0b18eb32e7da83d.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0c0b18eb32e7da83d.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-15-186.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec267e00a4086801bd60f1191c81f587,SystemUUID:ec267e00-a408-6801-bd60-f1191c81f587,BootID:7c624b34-f575-4c40-8226-aef6612a1a56,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:28.297: INFO: Logging kubelet events for node i-0c0b18eb32e7da83d Jan 9 06:44:28.441: INFO: Logging pods the kubelet thinks is on node i-0c0b18eb32e7da83d Jan 9 06:44:28.589: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-ft8kr started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:28.589: INFO: hostpath-client started at 2023-01-09 06:40:50 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container hostpath-client ready: false, restart count 0 Jan 9 06:44:28.589: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kpwrs started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:44:28.589: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kxqfw started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:28.589: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-ck9r4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:28.589: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-lknfk started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:28.589: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-cfps7 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:28.589: INFO: webserver-pod started at 2023-01-09 06:39:16 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container agnhost ready: false, restart count 0 Jan 9 06:44:28.589: INFO: netserver-3 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:28.589: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-njsp9 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:28.589: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-mwm9d started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:28.589: INFO: netserver-3 started at 2023-01-09 06:39:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:28.589: INFO: ebs-csi-node-m5kg7 started at 2023-01-09 06:25:34 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:28.589: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:28.589: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:28.589: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:28.589: INFO: cilium-vx7vq started at 2023-01-09 06:25:34 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:28.589: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:28.589: INFO: coredns-559769c974-md8gz started at 2023-01-09 06:25:51 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container coredns ready: true, restart count 0 Jan 9 06:44:28.589: INFO: coredns-autoscaler-7cb5c5b969-4vwwx started at 2023-01-09 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container autoscaler ready: true, restart count 0 Jan 9 06:44:28.589: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-sf7zm started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:28.589: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-twg96 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:28.589: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:29.107: INFO: Latency metrics for node i-0c0b18eb32e7da83d [DeferCleanup (Each)] [sig-cli] Kubectl client tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "kubectl-145" for this suite. �[38;5;243m01/09/23 06:44:29.107�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sNetworking\sGranular\sChecks\:\sPods\sshould\sfunction\sfor\snode\-pod\scommunication\:\shttp\s\[LinuxOnly\]\s\[NodeConformance\]\s\[Conformance\]$'
test/e2e/framework/network/utils.go:866 k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc00072a000, {0x75c6f5c, 0x9}, 0xc004cfda40) test/e2e/framework/network/utils.go:866 +0x1d0 k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc00072a000, 0x47?) test/e2e/framework/network/utils.go:763 +0x55 k8s.io/kubernetes/test/e2e/framework/network.NewCoreNetworkingTestConfig(0xc000d9e000, 0x1) test/e2e/framework/network/utils.go:144 +0xfb k8s.io/kubernetes/test/e2e/common/network.glob..func1.1.4() test/e2e/common/network/networking.go:106 +0x34from junit_01.xml
[BeforeEach] [sig-network] Networking set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/09/23 06:39:16.962�[0m Jan 9 06:39:16.962: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename pod-network-test �[38;5;243m01/09/23 06:39:16.963�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/09/23 06:39:17.392�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/09/23 06:39:17.675�[0m [BeforeEach] [sig-network] Networking test/e2e/framework/metrics/init/init.go:31 [It] should function for node-pod communication: http [LinuxOnly] [NodeConformance] [Conformance] test/e2e/common/network/networking.go:105 �[1mSTEP:�[0m Performing setup for networking test in namespace pod-network-test-9541 �[38;5;243m01/09/23 06:39:17.957�[0m �[1mSTEP:�[0m creating a selector �[38;5;243m01/09/23 06:39:17.958�[0m �[1mSTEP:�[0m Creating the service pods in kubernetes �[38;5;243m01/09/23 06:39:17.958�[0m Jan 9 06:39:17.958: INFO: Waiting up to 10m0s for all (but 0) nodes to be schedulable Jan 9 06:39:18.822: INFO: Waiting up to 5m0s for pod "netserver-0" in namespace "pod-network-test-9541" to be "running and ready" Jan 9 06:39:18.967: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 145.088554ms Jan 9 06:39:18.967: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:21.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2.289000369s Jan 9 06:39:21.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:23.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4.288267343s Jan 9 06:39:23.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:25.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 6.288034334s Jan 9 06:39:25.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:27.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 8.287585529s Jan 9 06:39:27.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:29.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 10.288066004s Jan 9 06:39:29.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:31.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 12.288378379s Jan 9 06:39:31.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:33.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 14.287269963s Jan 9 06:39:33.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:35.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 16.28727868s Jan 9 06:39:35.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:37.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 18.287681815s Jan 9 06:39:37.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:39.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 20.2876863s Jan 9 06:39:39.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:41.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 22.288197914s Jan 9 06:39:41.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:43.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 24.288180522s Jan 9 06:39:43.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:45.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 26.287972965s Jan 9 06:39:45.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:47.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 28.288028368s Jan 9 06:39:47.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:49.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 30.287642504s Jan 9 06:39:49.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:51.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 32.288248624s Jan 9 06:39:51.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:53.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 34.288268635s Jan 9 06:39:53.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:55.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 36.288325651s Jan 9 06:39:55.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:57.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 38.288424849s Jan 9 06:39:57.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:59.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 40.288563532s Jan 9 06:39:59.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:01.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 42.287503632s Jan 9 06:40:01.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:03.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 44.287705132s Jan 9 06:40:03.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:05.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 46.28741965s Jan 9 06:40:05.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:07.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 48.287742964s Jan 9 06:40:07.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:09.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 50.288110453s Jan 9 06:40:09.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:11.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 52.287640177s Jan 9 06:40:11.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:13.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 54.288463128s Jan 9 06:40:13.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:15.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 56.28809377s Jan 9 06:40:15.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:17.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 58.287357583s Jan 9 06:40:17.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:19.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.28806223s Jan 9 06:40:19.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:21.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.287215343s Jan 9 06:40:21.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:23.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.287282905s Jan 9 06:40:23.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:25.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.288311357s Jan 9 06:40:25.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:27.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.288405103s Jan 9 06:40:27.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:29.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.287192581s Jan 9 06:40:29.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:31.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.287135579s Jan 9 06:40:31.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:33.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.288277338s Jan 9 06:40:33.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:35.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.28759007s Jan 9 06:40:35.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:37.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.288074663s Jan 9 06:40:37.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:39.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.288030687s Jan 9 06:40:39.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:41.116: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.294252289s Jan 9 06:40:41.116: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:43.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.288311855s Jan 9 06:40:43.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:45.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.28835542s Jan 9 06:40:45.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:47.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.28803098s Jan 9 06:40:47.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:49.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.287297279s Jan 9 06:40:49.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:51.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.287934089s Jan 9 06:40:51.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:53.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.287934663s Jan 9 06:40:53.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:55.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.287381311s Jan 9 06:40:55.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:57.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.288434464s Jan 9 06:40:57.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:59.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.287439387s Jan 9 06:40:59.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:01.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.287975138s Jan 9 06:41:01.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:03.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.287621563s Jan 9 06:41:03.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:05.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.287195901s Jan 9 06:41:05.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:07.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.288804193s Jan 9 06:41:07.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:09.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.28805558s Jan 9 06:41:09.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:11.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.288056597s Jan 9 06:41:11.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:13.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.287876833s Jan 9 06:41:13.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:15.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.288519582s Jan 9 06:41:15.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:17.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.287296785s Jan 9 06:41:17.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:19.115: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.293319597s Jan 9 06:41:19.115: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:21.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.287926988s Jan 9 06:41:21.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:23.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.288179468s Jan 9 06:41:23.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:25.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.28724689s Jan 9 06:41:25.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:27.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.288352293s Jan 9 06:41:27.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:29.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.28786436s Jan 9 06:41:29.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:31.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.287720498s Jan 9 06:41:31.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:33.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.288145825s Jan 9 06:41:33.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:35.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.287937496s Jan 9 06:41:35.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:37.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.287781568s Jan 9 06:41:37.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:39.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.287487213s Jan 9 06:41:39.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:41.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.288173957s Jan 9 06:41:41.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:43.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.287986405s Jan 9 06:41:43.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:45.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.288197544s Jan 9 06:41:45.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:47.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.287556842s Jan 9 06:41:47.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:49.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.287626816s Jan 9 06:41:49.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:51.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.288270308s Jan 9 06:41:51.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:53.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.289398394s Jan 9 06:41:53.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:55.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.288694075s Jan 9 06:41:55.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:57.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.288623918s Jan 9 06:41:57.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:59.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.288461234s Jan 9 06:41:59.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:01.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.288386586s Jan 9 06:42:01.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:03.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.28791555s Jan 9 06:42:03.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:05.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.288870299s Jan 9 06:42:05.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:07.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.288407859s Jan 9 06:42:07.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:09.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.288306457s Jan 9 06:42:09.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:11.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.287350365s Jan 9 06:42:11.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:13.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.287818873s Jan 9 06:42:13.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:15.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.288154587s Jan 9 06:42:15.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:17.112: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.289581068s Jan 9 06:42:17.112: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:19.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.287452535s Jan 9 06:42:19.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:21.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.288341104s Jan 9 06:42:21.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:23.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.28803782s Jan 9 06:42:23.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:25.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.287766065s Jan 9 06:42:25.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:27.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.28786673s Jan 9 06:42:27.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:29.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.288536981s Jan 9 06:42:29.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:31.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.288235161s Jan 9 06:42:31.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:33.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.288399178s Jan 9 06:42:33.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:35.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.288274812s Jan 9 06:42:35.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:37.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.287557496s Jan 9 06:42:37.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:39.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.287649444s Jan 9 06:42:39.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:41.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.287235731s Jan 9 06:42:41.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:43.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.288534973s Jan 9 06:42:43.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:45.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.288158918s Jan 9 06:42:45.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:47.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.287443691s Jan 9 06:42:47.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:49.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.287687274s Jan 9 06:42:49.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:51.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.28807716s Jan 9 06:42:51.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:53.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.287572365s Jan 9 06:42:53.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:55.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.287835605s Jan 9 06:42:55.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:57.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.288605461s Jan 9 06:42:57.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:59.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.287630005s Jan 9 06:42:59.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:01.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.287873147s Jan 9 06:43:01.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:03.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.287388043s Jan 9 06:43:03.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:05.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.288567809s Jan 9 06:43:05.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:07.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.288455107s Jan 9 06:43:07.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:09.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.287578436s Jan 9 06:43:09.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:11.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.288306146s Jan 9 06:43:11.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:13.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.288349297s Jan 9 06:43:13.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:15.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.288051204s Jan 9 06:43:15.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:17.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.287961015s Jan 9 06:43:17.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:19.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.288423576s Jan 9 06:43:19.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:21.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.288524832s Jan 9 06:43:21.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:23.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.287908341s Jan 9 06:43:23.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:25.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.288360361s Jan 9 06:43:25.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:27.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.287740088s Jan 9 06:43:27.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:29.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.288151549s Jan 9 06:43:29.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:31.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.287080479s Jan 9 06:43:31.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:33.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.287870321s Jan 9 06:43:33.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:35.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.288489028s Jan 9 06:43:35.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:37.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.289317425s Jan 9 06:43:37.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:39.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.288150928s Jan 9 06:43:39.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:41.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.28803977s Jan 9 06:43:41.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:43.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.287538991s Jan 9 06:43:43.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:45.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.288281932s Jan 9 06:43:45.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:47.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.288101318s Jan 9 06:43:47.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:49.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.288222657s Jan 9 06:43:49.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:51.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.28798041s Jan 9 06:43:51.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:53.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.288548979s Jan 9 06:43:53.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:55.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.287761281s Jan 9 06:43:55.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:57.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.288445733s Jan 9 06:43:57.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:59.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.288081842s Jan 9 06:43:59.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:01.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.287979753s Jan 9 06:44:01.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:03.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.288751147s Jan 9 06:44:03.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:05.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.287504885s Jan 9 06:44:05.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:07.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.287375633s Jan 9 06:44:07.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:09.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.288111913s Jan 9 06:44:09.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:11.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.288310375s Jan 9 06:44:11.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:13.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.287590302s Jan 9 06:44:13.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:15.111: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.288516857s Jan 9 06:44:15.111: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:17.109: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.287168166s Jan 9 06:44:17.109: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:19.110: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.28826544s Jan 9 06:44:19.110: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:19.252: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.429991126s Jan 9 06:44:19.252: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:19.253: INFO: Unexpected error: <*pod.timeoutError | 0xc0039b1080>: { msg: "timed out while waiting for pod pod-network-test-9541/netserver-0 to be running and ready", observedObjects: [ <*v1.Pod | 0xc003cf4000>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "netserver-0", GenerateName: "", Namespace: "pod-network-test-9541", SelfLink: "", UID: "eb353103-d3fd-4024-b04e-412985d6553f", ResourceVersion: "27976", Generation: 0, CreationTimestamp: { Time: { wall: 0, ext: 63808843158, loc: { name: "Local", zone: [ {name: "UTC", offset: 0, isDST: false}, ], tx: [ { when: -576460752303423488, index: 0, isstd: false, isutc: false, }, ], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: "UTC", offset: 0, isDST: false}, }, }, }, DeletionTimestamp: nil, DeletionGracePeriodSeconds: nil, Labels: { "selector-8e005b52-20e8-4f79-82ea-1f394f4aed46": "true", }, Annotations: nil, OwnerReferences: nil, Finalizers: nil, ManagedFields: [ { Manager: "e2e.test", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63808843158, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:metadata\":{\"f:labels\":{\".\":{},\"f:selector-8e005b52-20e8-4f79-82ea-1f394f4aed46\":{}}},\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"webserver\\\"}\":{\".\":{},\"f:args\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:livenessProbe\":{\".\":{},\"f:failureThreshold\":{},\"f:httpGet\":{\".\":{},\"f:path\":{},\"f:port\":{},\"f:scheme\":{}},\"f:initialDelaySeconds\":{},\"f:periodSeconds\":{},\"f:successThreshold\":{},\"f:timeoutSeconds\":{}},\"f:name\":{},\"f:ports\":{\".\":{},\"k:{\\\"containerPort\\\":8081,\\\"protocol\\\":\\\"UDP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:name\":{},\"f:protocol\":{}},\"k:{\\\"containerPort\\\":8083,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:name\":{},\"f:protocol\":{}}},\"f:readinessProbe\":{\".\":{},\"f:failureThreshold\":{},\"f:httpGet\":{\".\":{},\"f:path\":{},\"f:port\":{},\"f:scheme\":{}},\"f:initialDelaySeconds\":{},\"f:periodSeconds\":{},\"f:successThreshold\":{},\"f:timeoutSeconds\":{}},\"f:resources\":{},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:nodeSelector\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:terminationGracePeriodSeconds\":{}}}", }, Subresource: "", }, { Manager: "kubelet", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63808843158, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}", }, Subresource: "status", }, ], }, Spec: { Volumes: [ { Name: "kube-api-access-wcr2l", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: nil, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: { Sources: [ { Secret: ..., DownwardAPI: ..., ConfigMap: ..., ServiceAccountToken: ..., }, ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Jan 9 06:44:19.253: FAIL: timed out while waiting for pod pod-network-test-9541/netserver-0 to be running and ready Full Stack Trace k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc00072a000, {0x75c6f5c, 0x9}, 0xc004cfda40) test/e2e/framework/network/utils.go:866 +0x1d0 k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc00072a000, 0x47?) test/e2e/framework/network/utils.go:763 +0x55 k8s.io/kubernetes/test/e2e/framework/network.NewCoreNetworkingTestConfig(0xc000d9e000, 0x1) test/e2e/framework/network/utils.go:144 +0xfb k8s.io/kubernetes/test/e2e/common/network.glob..func1.1.4() test/e2e/common/network/networking.go:106 +0x34 [AfterEach] [sig-network] Networking test/e2e/framework/node/init/init.go:32 Jan 9 06:44:19.254: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-network] Networking test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-network] Networking dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/09/23 06:44:19.399�[0m �[1mSTEP:�[0m Collecting events from namespace "pod-network-test-9541". �[38;5;243m01/09/23 06:44:19.399�[0m �[1mSTEP:�[0m Found 30 events. �[38;5;243m01/09/23 06:44:19.542�[0m Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:18 +0000 UTC - event for netserver-0: {default-scheduler } Scheduled: Successfully assigned pod-network-test-9541/netserver-0 to i-0585809501bfc393d Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:18 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "38ea150ea188f4cd04f831b71eab9ec6faaa850d39af81a249dd57a4978508dc": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:18 +0000 UTC - event for netserver-1: {default-scheduler } Scheduled: Successfully assigned pod-network-test-9541/netserver-1 to i-06d5855698aa7cb08 Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:18 +0000 UTC - event for netserver-2: {default-scheduler } Scheduled: Successfully assigned pod-network-test-9541/netserver-2 to i-087c2326a312024ef Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:18 +0000 UTC - event for netserver-3: {default-scheduler } Scheduled: Successfully assigned pod-network-test-9541/netserver-3 to i-0c0b18eb32e7da83d Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:19 +0000 UTC - event for netserver-1: {kubelet i-06d5855698aa7cb08} Created: Created container webserver Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:19 +0000 UTC - event for netserver-1: {kubelet i-06d5855698aa7cb08} Started: Started container webserver Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:19 +0000 UTC - event for netserver-1: {kubelet i-06d5855698aa7cb08} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:19 +0000 UTC - event for netserver-2: {kubelet i-087c2326a312024ef} Started: Started container webserver Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:19 +0000 UTC - event for netserver-2: {kubelet i-087c2326a312024ef} Created: Created container webserver Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:19 +0000 UTC - event for netserver-2: {kubelet i-087c2326a312024ef} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:19 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c2a109fc93beed2f3d69a4f1d212163e6e0eaaab92c0eb19956f94cbb5c5ec71": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:32 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f6d1bcb987c0286cba950b9f757702aa7b7928ca1d61f6721ed7e0ec8105ed09": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:32 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "933e89e4f17075e3ec7a1e80912f1a2e0013e989d74266a9f586bde0b9e261a3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:45 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4c20ab2c7a6d724ff89b55424e30239a3ee8b50f3f833a034990d58ca95b1726": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:46 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8b786c784420581393a49383ad4201cbb23cd985ce086b9318999496d5a7969a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:39:56 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "74da8e624ee9cab59adf07504b0bf51d6175443ef04b8cbe9784671745a05029": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:40:02 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "88cd3079e4f31b3f221240c480c59e995d145d50bbe6701d30ed4b431807d4dc": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:40:07 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "42471fa660dca0e9d561844c043c3a376c98edeec9720902d3edb7893d720f04": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:40:15 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b43a8c2f70133a5f7b2dbbba6c975bb6cfb5818532a78d1783447dc5b249defe": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:40:20 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f3eaac99163ee62b5ab5c9026d39044cb7a1994ca5f3a11df89e368e2cc50fdb": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:40:28 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1c6326ff60c34b0492554aaf6be79e483516ad3173e8f0fb426af6782ec61554": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:40:34 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9dc8d05626add61553ef955c6eff0069b851c90d07afbfdd7ddf31806ff00208": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:40:43 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f169365e7dc81794dceefc6dadc7cce3c83ad773e69adc7f5b4d66dc533a8907": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:40:45 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "092a702d31cc43426e5317708683afc5281ed36634e88ec719f3cd7376baada0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:40:56 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e31815435c36345e5a00c5f908275111939f416bd9d73b4e65aac16d5c0f2afb": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:40:59 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5361153b64312bc8ec3f2bb1115c845d75b1c7f978192186b4db909f6bcf5145": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:41:10 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4878e145172f001c0afae0c3aadfd4b0b3fefcf17a0cd522ba7afeae6cb9247e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:41:12 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c00a2802dc2598f324ebabf6455e165dcf968ad55883b44bd580a4cedc3e2afc": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.542: INFO: At 2023-01-09 06:41:25 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "fda6c612458995f61ab8f1ea805b20e202215d61d79538944f0c075fbc8e8563": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:19.684: INFO: POD NODE PHASE GRACE CONDITIONS Jan 9 06:44:19.684: INFO: netserver-0 i-0585809501bfc393d Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:18 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:18 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:18 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:18 +0000 UTC }] Jan 9 06:44:19.684: INFO: netserver-1 i-06d5855698aa7cb08 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:18 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:38 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:38 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:18 +0000 UTC }] Jan 9 06:44:19.684: INFO: netserver-2 i-087c2326a312024ef Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:18 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:38 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:38 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:18 +0000 UTC }] Jan 9 06:44:19.684: INFO: netserver-3 i-0c0b18eb32e7da83d Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:18 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:18 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:18 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:18 +0000 UTC }] Jan 9 06:44:19.684: INFO: Jan 9 06:44:19.828: INFO: Unable to fetch pod-network-test-9541/netserver-0/webserver logs: the server rejected our request for an unknown reason (get pods netserver-0) Jan 9 06:44:20.260: INFO: Unable to fetch pod-network-test-9541/netserver-3/webserver logs: the server rejected our request for an unknown reason (get pods netserver-3) Jan 9 06:44:20.403: INFO: Logging node info for node i-02b09d19b2096261b Jan 9 06:44:20.545: INFO: Node Info: &Node{ObjectMeta:{i-02b09d19b2096261b efa11f02-6d54-4902-bbeb-efecc1f02cd1 29640 0 2023-01-09 06:23:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-02b09d19b2096261b kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.12 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02b09d19b2096261b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-09 06:23:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-09 06:24:22 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-09 06:24:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-09 06:41:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-02b09d19b2096261b,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:24:43 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.12,},NodeAddress{Type:ExternalIP,Address:54.94.119.205,},NodeAddress{Type:InternalDNS,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-119-205.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2da51bd0e40b36975082ac186a369f,SystemUUID:ec2da51b-d0e4-0b36-9750-82ac186a369f,BootID:83891f80-d7aa-4da9-ab10-44706ab3b631,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43453930,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42803458,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967347,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:20.545: INFO: Logging kubelet events for node i-02b09d19b2096261b Jan 9 06:44:20.690: INFO: Logging pods the kubelet thinks is on node i-02b09d19b2096261b Jan 9 06:44:20.838: INFO: kube-controller-manager-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:20.838: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 9 06:44:20.838: INFO: kube-scheduler-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:20.838: INFO: Container kube-scheduler ready: true, restart count 0 Jan 9 06:44:20.838: INFO: etcd-manager-events-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:20.838: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:44:20.838: INFO: aws-cloud-controller-manager-kj4b6 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:20.838: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 9 06:44:20.838: INFO: kops-controller-8qjsv started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:20.838: INFO: Container kops-controller ready: true, restart count 0 Jan 9 06:44:20.838: INFO: ebs-csi-node-kblmh started at 2023-01-09 06:24:28 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:20.838: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:20.838: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:20.838: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:20.838: INFO: ebs-csi-controller-89b649c94-zwk75 started at 2023-01-09 06:24:28 +0000 UTC (0+5 container statuses recorded) Jan 9 06:44:20.838: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:44:20.838: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:44:20.838: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:44:20.838: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:20.838: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:20.838: INFO: etcd-manager-main-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:20.838: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:44:20.838: INFO: kube-apiserver-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+2 container statuses recorded) Jan 9 06:44:20.838: INFO: Container healthcheck ready: true, restart count 0 Jan 9 06:44:20.838: INFO: Container kube-apiserver ready: true, restart count 1 Jan 9 06:44:20.838: INFO: cilium-97g4h started at 2023-01-09 06:24:28 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:20.838: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:20.838: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:20.838: INFO: dns-controller-69987775c6-mv7h2 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:20.838: INFO: Container dns-controller ready: true, restart count 0 Jan 9 06:44:20.838: INFO: cilium-operator-5dd44dc49f-fgc7c started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:20.838: INFO: Container cilium-operator ready: true, restart count 0 Jan 9 06:44:21.342: INFO: Latency metrics for node i-02b09d19b2096261b Jan 9 06:44:21.342: INFO: Logging node info for node i-0585809501bfc393d Jan 9 06:44:21.484: INFO: Node Info: &Node{ObjectMeta:{i-0585809501bfc393d 8c7c1c77-e673-43a0-81a4-fcf97493bcd8 29271 0 2023-01-09 06:25:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0585809501bfc393d kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-0585809501bfc393d topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.147 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0585809501bfc393d"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-09 06:41:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0585809501bfc393d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.147,},NodeAddress{Type:ExternalIP,Address:54.94.242.160,},NodeAddress{Type:InternalDNS,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-242-160.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec206fc38cf93b7ea12ba838ad958aa3,SystemUUID:ec206fc3-8cf9-3b7e-a12b-a838ad958aa3,BootID:afb66846-6976-49bf-9e61-b54e5af8a3eb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:21.485: INFO: Logging kubelet events for node i-0585809501bfc393d Jan 9 06:44:21.629: INFO: Logging pods the kubelet thinks is on node i-0585809501bfc393d Jan 9 06:44:21.779: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-2jgj6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:21.779: INFO: pod-subpath-test-preprovisionedpv-wp87 started at 2023-01-09 06:39:43 +0000 UTC (2+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Init container init-volume-preprovisionedpv-wp87 ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Init container test-init-subpath-preprovisionedpv-wp87 ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Container test-container-subpath-preprovisionedpv-wp87 ready: false, restart count 0 Jan 9 06:44:21.779: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-cs2m5 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:21.779: INFO: test-rollover-controller-75rk6 started at 2023-01-09 06:39:32 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:21.779: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-97f28 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:21.779: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-p6pfg started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:21.779: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-6hzsb started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:21.779: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:40:25 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:21.779: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:21.779: INFO: hostexec-i-0585809501bfc393d-v4gr6 started at 2023-01-09 06:39:24 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:21.779: INFO: netserver-0 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:21.779: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-bxrs7 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:21.779: INFO: netserver-0 started at 2023-01-09 06:39:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:21.779: INFO: cilium-f5j8w started at 2023-01-09 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:21.779: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:21.779: INFO: webserver-7f5969cbc7-bm97p started at 2023-01-09 06:41:26 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:21.779: INFO: alpine-nnp-true-7168054a-3362-4368-bc5e-54ecf20fc5ca started at 2023-01-09 06:39:32 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container alpine-nnp-true-7168054a-3362-4368-bc5e-54ecf20fc5ca ready: false, restart count 0 Jan 9 06:44:21.779: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:39:17 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:21.779: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:21.779: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fs9d6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:21.779: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fm99c started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:21.779: INFO: ebs-csi-node-6djgn started at 2023-01-09 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:21.779: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:21.779: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:21.779: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:21.779: INFO: alpine-nnp-false-4247b683-138d-4632-a6a3-6d844097c353 started at 2023-01-09 06:41:15 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container alpine-nnp-false-4247b683-138d-4632-a6a3-6d844097c353 ready: false, restart count 0 Jan 9 06:44:21.779: INFO: webserver-66b8c99f6-ls7f4 started at 2023-01-09 06:41:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:21.779: INFO: execpodjqtvg started at 2023-01-09 06:42:07 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container agnhost-container ready: false, restart count 0 Jan 9 06:44:21.779: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-xtk85 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:21.779: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-qm6jm started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:21.779: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8rq57 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:21.779: INFO: update-demo-nautilus-gqbgn started at 2023-01-09 06:39:17 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.779: INFO: Container update-demo ready: false, restart count 0 Jan 9 06:44:21.779: INFO: dns-test-a96d67de-eeb2-4e87-8d66-17c313a4ef5a started at 2023-01-09 06:40:06 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:21.779: INFO: Container jessie-querier ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Container querier ready: false, restart count 0 Jan 9 06:44:21.779: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:22.397: INFO: Latency metrics for node i-0585809501bfc393d Jan 9 06:44:22.397: INFO: Logging node info for node i-06d5855698aa7cb08 Jan 9 06:44:22.539: INFO: Node Info: &Node{ObjectMeta:{i-06d5855698aa7cb08 bdfd8f9d-5aa1-4cff-806a-849d1c29fcb6 28608 0 2023-01-09 06:25:34 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06d5855698aa7cb08 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-06d5855698aa7cb08 topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.46.163 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-2682":"i-06d5855698aa7cb08","ebs.csi.aws.com":"i-06d5855698aa7cb08"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:39:47 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-09 06:39:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06d5855698aa7cb08,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.46.163,},NodeAddress{Type:ExternalIP,Address:18.228.235.107,},NodeAddress{Type:InternalDNS,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-107.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29e10a0890d67138c4eff5b197ee61,SystemUUID:ec29e10a-0890-d671-38c4-eff5b197ee61,BootID:9352599c-af91-4c75-860f-bb9a2789a998,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-2682^66d24229-8fe8-11ed-abbf-d2969ce11d4b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-2682^66d24229-8fe8-11ed-abbf-d2969ce11d4b,DevicePath:,},},Config:nil,},} Jan 9 06:44:22.539: INFO: Logging kubelet events for node i-06d5855698aa7cb08 Jan 9 06:44:22.684: INFO: Logging pods the kubelet thinks is on node i-06d5855698aa7cb08 Jan 9 06:44:22.835: INFO: webserver-7f5969cbc7-tz85c started at 2023-01-09 06:41:16 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:22.835: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-vwlp2 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.835: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pnbg6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.835: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-mkn6f started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.835: INFO: pod-fe7ddf40-2506-41f1-af53-09d0de5705e7 started at 2023-01-09 06:39:29 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container write-pod ready: false, restart count 0 Jan 9 06:44:22.835: INFO: netserver-1 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:22.835: INFO: ebs-csi-node-6dzh8 started at 2023-01-09 06:25:35 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:22.835: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:22.835: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:22.835: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:22.835: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:42:19 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:22.835: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:22.835: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:22.835: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:22.835: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:22.835: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:22.835: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:22.835: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:22.835: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:39:00 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:22.835: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:44:22.835: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:44:22.835: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:44:22.835: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 9 06:44:22.835: INFO: Container hostpath ready: true, restart count 0 Jan 9 06:44:22.835: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:22.835: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:22.835: INFO: netserver-1 started at 2023-01-09 06:39:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:22.835: INFO: csi-mockplugin-0 started at 2023-01-09 06:40:46 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:22.835: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:22.835: INFO: Container driver-registrar ready: false, restart count 0 Jan 9 06:44:22.835: INFO: Container mock ready: false, restart count 0 Jan 9 06:44:22.835: INFO: coredns-559769c974-qdhtp started at 2023-01-09 06:26:21 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container coredns ready: true, restart count 0 Jan 9 06:44:22.835: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fncl4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.835: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8hxw6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.835: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-wg2gn started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.835: INFO: pod-d7eb61ad-d823-42ce-b057-d1397e0091be started at 2023-01-09 06:39:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container write-pod ready: false, restart count 0 Jan 9 06:44:22.835: INFO: cilium-wbn8d started at 2023-01-09 06:25:35 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:22.835: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:22.835: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-hpb4w started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.835: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kr2c4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.835: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-b42jp started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.835: INFO: hostexec-i-06d5855698aa7cb08-gdgwr started at 2023-01-09 06:39:24 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.835: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:23.387: INFO: Latency metrics for node i-06d5855698aa7cb08 Jan 9 06:44:23.387: INFO: Logging node info for node i-087c2326a312024ef Jan 9 06:44:23.529: INFO: Node Info: &Node{ObjectMeta:{i-087c2326a312024ef 45db8cc8-250b-411a-b344-d6bbffe46b7d 29314 0 2023-01-09 06:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-087c2326a312024ef kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-087c2326a312024ef topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.218 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-087c2326a312024ef"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:40:12 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-09 06:41:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-087c2326a312024ef,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.218,},NodeAddress{Type:ExternalIP,Address:18.230.17.62,},NodeAddress{Type:InternalDNS,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-17-62.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec28fbdf9f29233021716214b9f8da3a,SystemUUID:ec28fbdf-9f29-2330-2171-6214b9f8da3a,BootID:4dbf8c75-8514-419f-b443-1a44f89f420f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d49342c84d2a787d],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d49342c84d2a787d,DevicePath:,},},Config:nil,},} Jan 9 06:44:23.529: INFO: Logging kubelet events for node i-087c2326a312024ef Jan 9 06:44:23.674: INFO: Logging pods the kubelet thinks is on node i-087c2326a312024ef Jan 9 06:44:23.825: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-b8q64 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.825: INFO: test-rs-gj54c started at 2023-01-09 06:44:23 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:23.825: INFO: oidc-discovery-validator started at 2023-01-09 06:39:17 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container oidc-discovery-validator ready: false, restart count 0 Jan 9 06:44:23.825: INFO: webserver-66b8c99f6-gt295 started at 2023-01-09 06:41:30 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:23.825: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-85kn4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.825: INFO: webserver-7f5969cbc7-rf7jj started at 2023-01-09 06:41:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:23.825: INFO: pod1 started at 2023-01-09 06:40:38 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:23.825: INFO: sample-webhook-deployment-865554f4d9-sqdbc started at 2023-01-09 06:44:13 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container sample-webhook ready: false, restart count 0 Jan 9 06:44:23.825: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-dlhg8 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:44:23.825: INFO: local-client started at 2023-01-09 06:42:01 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container local-client ready: false, restart count 0 Jan 9 06:44:23.825: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-h8zj6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.825: INFO: netserver-2 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:23.825: INFO: ebs-csi-node-jvd2b started at 2023-01-09 06:25:35 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:23.825: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:23.825: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:23.825: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:23.825: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-plbnr started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.825: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pvc5d started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:44:23.825: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-r89ft started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.825: INFO: cilium-gw6mm started at 2023-01-09 06:25:35 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:23.825: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:23.825: INFO: hostexec-i-087c2326a312024ef-2vcfq started at 2023-01-09 06:40:20 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:23.825: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pcnbb started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.825: INFO: netserver-2 started at 2023-01-09 06:39:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:23.825: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8brjv started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.825: INFO: pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d started at 2023-01-09 06:39:05 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container write-pod ready: false, restart count 0 Jan 9 06:44:23.825: INFO: test-pod-72932517-45d4-49a0-93e1-49d6ad3c30db started at 2023-01-09 06:39:32 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container agnhost-container ready: false, restart count 0 Jan 9 06:44:23.825: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-qxtwd started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.825: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:24.370: INFO: Latency metrics for node i-087c2326a312024ef Jan 9 06:44:24.370: INFO: Logging node info for node i-0c0b18eb32e7da83d Jan 9 06:44:24.512: INFO: Node Info: &Node{ObjectMeta:{i-0c0b18eb32e7da83d f1550edf-43cd-451c-bf01-eb92d6980d45 30343 0 2023-01-09 06:25:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0c0b18eb32e7da83d kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-0c0b18eb32e7da83d topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.55.27 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0c0b18eb32e7da83d"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-09 06:44:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c0b18eb32e7da83d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.55.27,},NodeAddress{Type:ExternalIP,Address:18.228.15.186,},NodeAddress{Type:InternalDNS,Address:i-0c0b18eb32e7da83d.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0c0b18eb32e7da83d.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-15-186.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec267e00a4086801bd60f1191c81f587,SystemUUID:ec267e00-a408-6801-bd60-f1191c81f587,BootID:7c624b34-f575-4c40-8226-aef6612a1a56,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:24.512: INFO: Logging kubelet events for node i-0c0b18eb32e7da83d Jan 9 06:44:24.656: INFO: Logging pods the kubelet thinks is on node i-0c0b18eb32e7da83d Jan 9 06:44:24.803: INFO: hostpath-client started at 2023-01-09 06:40:50 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.803: INFO: Container hostpath-client ready: false, restart count 0 Jan 9 06:44:24.803: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kpwrs started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.803: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:44:24.803: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kxqfw started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.803: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:24.803: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-ck9r4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.803: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:24.803: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-lknfk started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.803: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:24.803: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-cfps7 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.803: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:24.803: INFO: webserver-pod started at 2023-01-09 06:39:16 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.803: INFO: Container agnhost ready: false, restart count 0 Jan 9 06:44:24.803: INFO: netserver-3 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.803: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:24.803: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-njsp9 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.803: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:24.803: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-sf7zm started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.803: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:24.803: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-mwm9d started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.803: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:24.803: INFO: netserver-3 started at 2023-01-09 06:39:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.803: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:24.803: INFO: ebs-csi-node-m5kg7 started at 2023-01-09 06:25:34 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:24.803: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:24.803: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:24.803: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:24.804: INFO: cilium-vx7vq started at 2023-01-09 06:25:34 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:24.804: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:24.804: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:24.804: INFO: coredns-559769c974-md8gz started at 2023-01-09 06:25:51 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.804: INFO: Container coredns ready: true, restart count 0 Jan 9 06:44:24.804: INFO: coredns-autoscaler-7cb5c5b969-4vwwx started at 2023-01-09 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.804: INFO: Container autoscaler ready: true, restart count 0 Jan 9 06:44:24.804: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-twg96 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.804: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:24.804: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-ft8kr started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:24.804: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:25.313: INFO: Latency metrics for node i-0c0b18eb32e7da83d [DeferCleanup (Each)] [sig-network] Networking tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "pod-network-test-9541" for this suite. �[38;5;243m01/09/23 06:44:25.313�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sNetworking\sGranular\sChecks\:\sServices\sshould\sfunction\sfor\sclient\sIP\sbased\ssession\saffinity\:\shttp\s\[LinuxOnly\]$'
test/e2e/framework/network/utils.go:866 k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc000314b60, {0x75c6f5c, 0x9}, 0xc004547ce0) test/e2e/framework/network/utils.go:866 +0x1d0 k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc000314b60, 0x7f803ab581a8?) test/e2e/framework/network/utils.go:763 +0x55 k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000314b60, 0x3e?) test/e2e/framework/network/utils.go:778 +0x3e k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc000cbc5a0, {0x0, 0x0, 0xc001306198?}) test/e2e/framework/network/utils.go:131 +0x125 k8s.io/kubernetes/test/e2e/network.glob..func22.6.16() test/e2e/network/networking.go:417 +0x36from junit_01.xml
[BeforeEach] [sig-network] Networking set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/09/23 06:39:31.877�[0m Jan 9 06:39:31.877: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename nettest �[38;5;243m01/09/23 06:39:31.878�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/09/23 06:39:32.308�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/09/23 06:39:32.592�[0m [BeforeEach] [sig-network] Networking test/e2e/framework/metrics/init/init.go:31 [It] should function for client IP based session affinity: http [LinuxOnly] test/e2e/network/networking.go:416 �[1mSTEP:�[0m Performing setup for networking test in namespace nettest-1377 �[38;5;243m01/09/23 06:39:32.877�[0m �[1mSTEP:�[0m creating a selector �[38;5;243m01/09/23 06:39:32.877�[0m �[1mSTEP:�[0m Creating the service pods in kubernetes �[38;5;243m01/09/23 06:39:32.877�[0m Jan 9 06:39:32.877: INFO: Waiting up to 10m0s for all (but 0) nodes to be schedulable Jan 9 06:39:33.747: INFO: Waiting up to 5m0s for pod "netserver-0" in namespace "nettest-1377" to be "running and ready" Jan 9 06:39:33.890: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 142.477649ms Jan 9 06:39:33.890: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:36.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2.285934147s Jan 9 06:39:36.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:38.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4.286141434s Jan 9 06:39:38.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:40.034: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 6.286768232s Jan 9 06:39:40.034: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:42.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 8.286105532s Jan 9 06:39:42.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:44.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 10.286113113s Jan 9 06:39:44.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:46.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 12.285448728s Jan 9 06:39:46.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:48.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 14.285937585s Jan 9 06:39:48.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:50.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 16.285669622s Jan 9 06:39:50.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:52.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 18.286296226s Jan 9 06:39:52.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:54.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 20.286209729s Jan 9 06:39:54.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:56.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 22.286170628s Jan 9 06:39:56.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:39:58.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 24.286062343s Jan 9 06:39:58.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:00.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 26.286211506s Jan 9 06:40:00.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:02.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 28.286306218s Jan 9 06:40:02.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:04.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 30.286156489s Jan 9 06:40:04.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:06.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 32.286029964s Jan 9 06:40:06.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:08.035: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 34.287654966s Jan 9 06:40:08.035: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:10.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 36.285971115s Jan 9 06:40:10.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:12.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 38.286090598s Jan 9 06:40:12.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:14.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 40.285409645s Jan 9 06:40:14.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:16.033: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 42.286231254s Jan 9 06:40:16.033: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:18.034: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 44.286771454s Jan 9 06:40:18.034: INFO: The phase of Pod netserver-0 is Running (Ready = false) Jan 9 06:40:20.034: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 46.286492026s Jan 9 06:40:20.034: INFO: The phase of Pod netserver-0 is Running (Ready = false) Jan 9 06:40:22.033: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 48.286097134s Jan 9 06:40:22.033: INFO: The phase of Pod netserver-0 is Running (Ready = false) Jan 9 06:40:24.033: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 50.286268415s Jan 9 06:40:24.033: INFO: The phase of Pod netserver-0 is Running (Ready = false) Jan 9 06:40:26.033: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 52.286116989s Jan 9 06:40:26.033: INFO: The phase of Pod netserver-0 is Running (Ready = false) Jan 9 06:40:28.033: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 54.285956603s Jan 9 06:40:28.033: INFO: The phase of Pod netserver-0 is Running (Ready = false) Jan 9 06:40:30.033: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 56.285968624s Jan 9 06:40:30.033: INFO: The phase of Pod netserver-0 is Running (Ready = false) Jan 9 06:40:32.034: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 58.286361323s Jan 9 06:40:32.034: INFO: The phase of Pod netserver-0 is Running (Ready = false) Jan 9 06:40:34.034: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 1m0.28662905s Jan 9 06:40:34.034: INFO: The phase of Pod netserver-0 is Running (Ready = false) Jan 9 06:40:36.033: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=true. Elapsed: 1m2.285811702s Jan 9 06:40:36.033: INFO: The phase of Pod netserver-0 is Running (Ready = true) Jan 9 06:40:36.033: INFO: Pod "netserver-0" satisfied condition "running and ready" Jan 9 06:40:36.175: INFO: Waiting up to 5m0s for pod "netserver-1" in namespace "nettest-1377" to be "running and ready" Jan 9 06:40:36.318: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 142.737195ms Jan 9 06:40:36.318: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:38.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2.286435993s Jan 9 06:40:38.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:40.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4.286889436s Jan 9 06:40:40.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:42.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 6.28625968s Jan 9 06:40:42.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:44.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 8.286464792s Jan 9 06:40:44.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:46.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 10.286169243s Jan 9 06:40:46.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:48.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 12.285924796s Jan 9 06:40:48.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:50.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 14.285627794s Jan 9 06:40:50.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:52.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 16.285720253s Jan 9 06:40:52.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:54.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 18.285915506s Jan 9 06:40:54.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:56.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 20.286604021s Jan 9 06:40:56.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:40:58.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 22.28632191s Jan 9 06:40:58.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:00.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 24.286638218s Jan 9 06:41:00.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:02.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 26.286768206s Jan 9 06:41:02.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:04.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 28.286677832s Jan 9 06:41:04.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:06.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 30.286597127s Jan 9 06:41:06.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:08.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 32.286108817s Jan 9 06:41:08.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:10.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 34.285966134s Jan 9 06:41:10.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:12.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 36.286262s Jan 9 06:41:12.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:14.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 38.285686666s Jan 9 06:41:14.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:16.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 40.286618068s Jan 9 06:41:16.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:18.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 42.286376372s Jan 9 06:41:18.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:20.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 44.286380242s Jan 9 06:41:20.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:22.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 46.28661191s Jan 9 06:41:22.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:24.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 48.286443089s Jan 9 06:41:24.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:26.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 50.286022148s Jan 9 06:41:26.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:28.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 52.286264806s Jan 9 06:41:28.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:30.463: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 54.287436804s Jan 9 06:41:30.463: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:32.463: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 56.287710508s Jan 9 06:41:32.463: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:34.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 58.286045462s Jan 9 06:41:34.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:36.463: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.287010835s Jan 9 06:41:36.463: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:38.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.286216576s Jan 9 06:41:38.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:40.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.286365258s Jan 9 06:41:40.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:42.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.286478692s Jan 9 06:41:42.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:44.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.286384858s Jan 9 06:41:44.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:46.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.286251258s Jan 9 06:41:46.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:48.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.286306021s Jan 9 06:41:48.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:50.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.286002184s Jan 9 06:41:50.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:52.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.285361385s Jan 9 06:41:52.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:54.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.286525875s Jan 9 06:41:54.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:56.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.285868282s Jan 9 06:41:56.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:41:58.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.286135303s Jan 9 06:41:58.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:00.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.286192797s Jan 9 06:42:00.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:02.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.286671445s Jan 9 06:42:02.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:04.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.286548533s Jan 9 06:42:04.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:06.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.286319352s Jan 9 06:42:06.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:08.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.285951056s Jan 9 06:42:08.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:10.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.286510546s Jan 9 06:42:10.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:12.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.28551136s Jan 9 06:42:12.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:14.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.2863616s Jan 9 06:42:14.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:16.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.286651334s Jan 9 06:42:16.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:18.463: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.287876324s Jan 9 06:42:18.463: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:20.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.286842773s Jan 9 06:42:20.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:22.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.28605414s Jan 9 06:42:22.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:24.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.28609407s Jan 9 06:42:24.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:26.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.286829504s Jan 9 06:42:26.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:28.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.286147206s Jan 9 06:42:28.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:30.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.286839044s Jan 9 06:42:30.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:32.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.286552759s Jan 9 06:42:32.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:34.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.286604606s Jan 9 06:42:34.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:36.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.286357506s Jan 9 06:42:36.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:38.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.286439213s Jan 9 06:42:38.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:40.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.28622914s Jan 9 06:42:40.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:42.466: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.290931257s Jan 9 06:42:42.466: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:44.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.28609439s Jan 9 06:42:44.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:46.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.285560099s Jan 9 06:42:46.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:48.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.286242663s Jan 9 06:42:48.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:50.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.286344411s Jan 9 06:42:50.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:52.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.286533466s Jan 9 06:42:52.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:54.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.285364286s Jan 9 06:42:54.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:56.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.286737605s Jan 9 06:42:56.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:42:58.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.286213927s Jan 9 06:42:58.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:00.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.28640261s Jan 9 06:43:00.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:02.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.286146696s Jan 9 06:43:02.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:04.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.286684205s Jan 9 06:43:04.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:06.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.285474176s Jan 9 06:43:06.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:08.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.286101097s Jan 9 06:43:08.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:10.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.285497409s Jan 9 06:43:10.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:12.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.28650321s Jan 9 06:43:12.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:14.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.286343405s Jan 9 06:43:14.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:16.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.286459455s Jan 9 06:43:16.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:18.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.286285972s Jan 9 06:43:18.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:20.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.286460683s Jan 9 06:43:20.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:22.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.286165519s Jan 9 06:43:22.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:24.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.286480301s Jan 9 06:43:24.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:26.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.286353274s Jan 9 06:43:26.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:28.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.286491555s Jan 9 06:43:28.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:30.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.285441491s Jan 9 06:43:30.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:32.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.286047404s Jan 9 06:43:32.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:34.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.286156318s Jan 9 06:43:34.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:36.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.286365295s Jan 9 06:43:36.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:38.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.285849181s Jan 9 06:43:38.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:40.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.286880192s Jan 9 06:43:40.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:42.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.286302823s Jan 9 06:43:42.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:44.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.285817606s Jan 9 06:43:44.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:46.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.286270038s Jan 9 06:43:46.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:48.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.28567483s Jan 9 06:43:48.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:50.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.285597745s Jan 9 06:43:50.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:52.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.286482581s Jan 9 06:43:52.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:54.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.286449033s Jan 9 06:43:54.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:56.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.286206834s Jan 9 06:43:56.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:43:58.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.285750812s Jan 9 06:43:58.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:00.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.285516478s Jan 9 06:44:00.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:02.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.286616684s Jan 9 06:44:02.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:04.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.28630565s Jan 9 06:44:04.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:06.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.286082647s Jan 9 06:44:06.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:08.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.285376007s Jan 9 06:44:08.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:10.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.286559148s Jan 9 06:44:10.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:12.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.286422639s Jan 9 06:44:12.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:14.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.285512977s Jan 9 06:44:14.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:16.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.28633603s Jan 9 06:44:16.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:18.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.285825134s Jan 9 06:44:18.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:20.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.285368638s Jan 9 06:44:20.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:22.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.286036688s Jan 9 06:44:22.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:24.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.285906258s Jan 9 06:44:24.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:26.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.285302741s Jan 9 06:44:26.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:28.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.286072899s Jan 9 06:44:28.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:30.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.285475541s Jan 9 06:44:30.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:32.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.28676508s Jan 9 06:44:32.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:34.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.286029229s Jan 9 06:44:34.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:36.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.286034508s Jan 9 06:44:36.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:38.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.286392775s Jan 9 06:44:38.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:40.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.286426299s Jan 9 06:44:40.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:42.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.28603551s Jan 9 06:44:42.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:44.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.286358019s Jan 9 06:44:44.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:46.468: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.292424185s Jan 9 06:44:46.468: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:48.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.286005497s Jan 9 06:44:48.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:50.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.285762337s Jan 9 06:44:50.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:52.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.2862907s Jan 9 06:44:52.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:54.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.286406787s Jan 9 06:44:54.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:56.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.286496817s Jan 9 06:44:56.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:44:58.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.285823896s Jan 9 06:44:58.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:00.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.286482509s Jan 9 06:45:00.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:02.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.286470096s Jan 9 06:45:02.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:04.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.286864554s Jan 9 06:45:04.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:06.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.286340709s Jan 9 06:45:06.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:08.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.285699418s Jan 9 06:45:08.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:10.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.285448942s Jan 9 06:45:10.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:12.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.286500679s Jan 9 06:45:12.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:14.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.285786465s Jan 9 06:45:14.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:16.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.286618637s Jan 9 06:45:16.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:18.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.286010786s Jan 9 06:45:18.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:20.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.286664937s Jan 9 06:45:20.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:22.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.286525384s Jan 9 06:45:22.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:24.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.285986407s Jan 9 06:45:24.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:26.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.285471097s Jan 9 06:45:26.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:28.461: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.285333724s Jan 9 06:45:28.461: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:30.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.286280375s Jan 9 06:45:30.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:32.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.286185962s Jan 9 06:45:32.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:34.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.286129343s Jan 9 06:45:34.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:36.462: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.286352978s Jan 9 06:45:36.462: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:36.604: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.428974466s Jan 9 06:45:36.604: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true) Jan 9 06:45:36.606: INFO: Unexpected error: <*pod.timeoutError | 0xc0040ad260>: { msg: "timed out while waiting for pod nettest-1377/netserver-1 to be running and ready", observedObjects: [ <*v1.Pod | 0xc0030d0480>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "netserver-1", GenerateName: "", Namespace: "nettest-1377", SelfLink: "", UID: "ff3e733b-8511-4162-976f-f997671ba0f6", ResourceVersion: "28429", Generation: 0, CreationTimestamp: { Time: { wall: 0, ext: 63808843173, loc: { name: "Local", zone: [ {name: "UTC", offset: 0, isDST: false}, ], tx: [ { when: -576460752303423488, index: 0, isstd: false, isutc: false, }, ], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: "UTC", offset: 0, isDST: false}, }, }, }, DeletionTimestamp: nil, DeletionGracePeriodSeconds: nil, Labels: { "selector-d5073834-bde4-4b3d-a2b1-8cdbcbcd1151": "true", }, Annotations: nil, OwnerReferences: nil, Finalizers: nil, ManagedFields: [ { Manager: "e2e.test", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63808843173, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:metadata\":{\"f:labels\":{\".\":{},\"f:selector-d5073834-bde4-4b3d-a2b1-8cdbcbcd1151\":{}}},\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"webserver\\\"}\":{\".\":{},\"f:args\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:livenessProbe\":{\".\":{},\"f:failureThreshold\":{},\"f:httpGet\":{\".\":{},\"f:path\":{},\"f:port\":{},\"f:scheme\":{}},\"f:initialDelaySeconds\":{},\"f:periodSeconds\":{},\"f:successThreshold\":{},\"f:timeoutSeconds\":{}},\"f:name\":{},\"f:ports\":{\".\":{},\"k:{\\\"containerPort\\\":8081,\\\"protocol\\\":\\\"UDP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:name\":{},\"f:protocol\":{}},\"k:{\\\"containerPort\\\":8083,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:name\":{},\"f:protocol\":{}}},\"f:readinessProbe\":{\".\":{},\"f:failureThreshold\":{},\"f:httpGet\":{\".\":{},\"f:path\":{},\"f:port\":{},\"f:scheme\":{}},\"f:initialDelaySeconds\":{},\"f:periodSeconds\":{},\"f:successThreshold\":{},\"f:timeoutSeconds\":{}},\"f:resources\":{},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:nodeSelector\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:terminationGracePeriodSeconds\":{}}}", }, Subresource: "", }, { Manager: "kubelet", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63808843173, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}", }, Subresource: "status", }, ], }, Spec: { Volumes: [ { Name: "kube-api-access-5vx4s", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: nil, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: { Sources: [ { Secret: ..., DownwardAPI: ..., ConfigMap: ..., ServiceAccountToken: ..., }, ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Jan 9 06:45:36.606: FAIL: timed out while waiting for pod nettest-1377/netserver-1 to be running and ready Full Stack Trace k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc000314b60, {0x75c6f5c, 0x9}, 0xc004547ce0) test/e2e/framework/network/utils.go:866 +0x1d0 k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc000314b60, 0x7f803ab581a8?) test/e2e/framework/network/utils.go:763 +0x55 k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000314b60, 0x3e?) test/e2e/framework/network/utils.go:778 +0x3e k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc000cbc5a0, {0x0, 0x0, 0xc001306198?}) test/e2e/framework/network/utils.go:131 +0x125 k8s.io/kubernetes/test/e2e/network.glob..func22.6.16() test/e2e/network/networking.go:417 +0x36 [AfterEach] [sig-network] Networking test/e2e/framework/node/init/init.go:32 Jan 9 06:45:36.607: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-network] Networking test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-network] Networking dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/09/23 06:45:36.75�[0m �[1mSTEP:�[0m Collecting events from namespace "nettest-1377". �[38;5;243m01/09/23 06:45:36.75�[0m �[1mSTEP:�[0m Found 33 events. �[38;5;243m01/09/23 06:45:36.894�[0m Jan 9 06:45:36.894: INFO: At 2023-01-09 06:39:33 +0000 UTC - event for netserver-0: {default-scheduler } Scheduled: Successfully assigned nettest-1377/netserver-0 to i-0585809501bfc393d Jan 9 06:45:36.894: INFO: At 2023-01-09 06:39:33 +0000 UTC - event for netserver-1: {default-scheduler } Scheduled: Successfully assigned nettest-1377/netserver-1 to i-06d5855698aa7cb08 Jan 9 06:45:36.894: INFO: At 2023-01-09 06:39:33 +0000 UTC - event for netserver-1: {kubelet i-06d5855698aa7cb08} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a20e7b3a6815694a8e947cc37b309a3e05ecd98a2ed97985f42a05a2710c0d43": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:39:33 +0000 UTC - event for netserver-2: {default-scheduler } Scheduled: Successfully assigned nettest-1377/netserver-2 to i-087c2326a312024ef Jan 9 06:45:36.894: INFO: At 2023-01-09 06:39:33 +0000 UTC - event for netserver-3: {default-scheduler } Scheduled: Successfully assigned nettest-1377/netserver-3 to i-0c0b18eb32e7da83d Jan 9 06:45:36.894: INFO: At 2023-01-09 06:39:34 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "671656cf8c4c586cc602b27435f0d1a71b495a773da46cb6055e8c978c0dc883": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:39:34 +0000 UTC - event for netserver-2: {kubelet i-087c2326a312024ef} Started: Started container webserver Jan 9 06:45:36.894: INFO: At 2023-01-09 06:39:34 +0000 UTC - event for netserver-2: {kubelet i-087c2326a312024ef} Created: Created container webserver Jan 9 06:45:36.894: INFO: At 2023-01-09 06:39:34 +0000 UTC - event for netserver-2: {kubelet i-087c2326a312024ef} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Jan 9 06:45:36.894: INFO: At 2023-01-09 06:39:34 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9d757b5ad81ea7293ef8ab34a6dff9a75f8e75cfa2226aca9564362ada78e60c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:39:47 +0000 UTC - event for netserver-1: {kubelet i-06d5855698aa7cb08} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1229714a369d2e2387334b341882ca7c8cb4f1176207d0bfa53868eaa303c9b6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:39:48 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6abfe10cd031568bd76286593f2fa71c40aeaf60522bce4ab529119fc7525ce7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:39:49 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "2f26a1c946d868c0b7a520b850820f8d5976ecfb746223d7b8d0bab76cd4d539": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:39:58 +0000 UTC - event for netserver-1: {kubelet i-06d5855698aa7cb08} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6c28d4aa8294e9f474fb56079efcb09d8436efc422e6770bd69dc249a0f783f0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:39:59 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9d2c33b0ca9493c94e06696311c999981a06a50abfacb75f9eb4e8f7b19394cc": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:40:03 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f7a32d004830de17dcc65800986785c1737fab93fdfe637cd4562b28c7f72fd9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:40:11 +0000 UTC - event for netserver-1: {kubelet i-06d5855698aa7cb08} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "83980129d8facd2d15fdbd295c786a2167d8ba044998c2a181eb6741111d8fae": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:40:13 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "63f72cec23cc8e6760c4de4b5afbba6fd4e3cdb8047a41ea6a53a9c8802ad445": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:40:16 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Jan 9 06:45:36.894: INFO: At 2023-01-09 06:40:16 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} Created: Created container webserver Jan 9 06:45:36.894: INFO: At 2023-01-09 06:40:17 +0000 UTC - event for netserver-0: {kubelet i-0585809501bfc393d} Started: Started container webserver Jan 9 06:45:36.894: INFO: At 2023-01-09 06:40:23 +0000 UTC - event for netserver-1: {kubelet i-06d5855698aa7cb08} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "662a9bbcd3d3cf209815a043cfadda0d833b4c798e71dcf98520aa6acaa7f04f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:40:27 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "fa2ff904e857cb288fad00b6f20a413eddbebb85d42516de5245c45f0866fd85": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:40:37 +0000 UTC - event for netserver-1: {kubelet i-06d5855698aa7cb08} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a82b4027692c646041f0af1e30bb51e6bc53d752c000bcde46b8c0d14ed8efb7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:40:41 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a8e438809d61a15c421f3301d53e9799a71b6c0e347075bec8af62a7db965987": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:40:50 +0000 UTC - event for netserver-1: {kubelet i-06d5855698aa7cb08} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "282f6ab90b9c1360440dcdf76aaf1a84a935a8473a94de36b6ad50f506193ee7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:40:52 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "816435c6655221f2c50e82742158644a281bba0a8474970a6b72452a1d25d51c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:41:01 +0000 UTC - event for netserver-1: {kubelet i-06d5855698aa7cb08} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "47d41f481160f8cb92deb4907c9fcaa91f9cc388280ff6decddcb8d36e90a1fc": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:41:05 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bd900467433f0efd3fad6a424883aeb26a0c956daf5e437fb097d823c194b734": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:41:17 +0000 UTC - event for netserver-1: {kubelet i-06d5855698aa7cb08} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b5b30a3684e08e35b453a817886d51c8c0cf77855498ec70375f0274ff16dd68": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:41:18 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "400208c0d5f556473d1ee1e6699e81be23d71b3986ec0011f312b45a6f532166": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:41:32 +0000 UTC - event for netserver-3: {kubelet i-0c0b18eb32e7da83d} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "35af0b542501b3c3ab03a92512067a6973a7c0f3863cd724b39951cd549cc14f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:36.894: INFO: At 2023-01-09 06:41:33 +0000 UTC - event for netserver-1: {kubelet i-06d5855698aa7cb08} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "687464d6d08382e619c35f93ab5cc2131072c27aa9766410b6d06f8e2d52be3f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:45:37.037: INFO: POD NODE PHASE GRACE CONDITIONS Jan 9 06:45:37.037: INFO: netserver-0 i-0585809501bfc393d Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:33 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:40:34 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:40:34 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:33 +0000 UTC }] Jan 9 06:45:37.037: INFO: netserver-1 i-06d5855698aa7cb08 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:33 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:33 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:33 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:33 +0000 UTC }] Jan 9 06:45:37.037: INFO: netserver-2 i-087c2326a312024ef Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:33 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:53 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:53 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:33 +0000 UTC }] Jan 9 06:45:37.037: INFO: netserver-3 i-0c0b18eb32e7da83d Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:33 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:33 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:33 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-09 06:39:33 +0000 UTC }] Jan 9 06:45:37.037: INFO: Jan 9 06:45:37.327: INFO: Unable to fetch nettest-1377/netserver-1/webserver logs: the server rejected our request for an unknown reason (get pods netserver-1) Jan 9 06:45:37.616: INFO: Unable to fetch nettest-1377/netserver-3/webserver logs: the server rejected our request for an unknown reason (get pods netserver-3) Jan 9 06:45:37.760: INFO: Logging node info for node i-02b09d19b2096261b Jan 9 06:45:37.902: INFO: Node Info: &Node{ObjectMeta:{i-02b09d19b2096261b efa11f02-6d54-4902-bbeb-efecc1f02cd1 29640 0 2023-01-09 06:23:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-02b09d19b2096261b kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.12 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02b09d19b2096261b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-09 06:23:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-09 06:24:22 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-09 06:24:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-09 06:41:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-02b09d19b2096261b,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:24:43 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.12,},NodeAddress{Type:ExternalIP,Address:54.94.119.205,},NodeAddress{Type:InternalDNS,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-119-205.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2da51bd0e40b36975082ac186a369f,SystemUUID:ec2da51b-d0e4-0b36-9750-82ac186a369f,BootID:83891f80-d7aa-4da9-ab10-44706ab3b631,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43453930,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42803458,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967347,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:45:37.903: INFO: Logging kubelet events for node i-02b09d19b2096261b Jan 9 06:45:38.048: INFO: Logging pods the kubelet thinks is on node i-02b09d19b2096261b Jan 9 06:45:38.197: INFO: dns-controller-69987775c6-mv7h2 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:38.197: INFO: Container dns-controller ready: true, restart count 0 Jan 9 06:45:38.197: INFO: cilium-operator-5dd44dc49f-fgc7c started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:38.197: INFO: Container cilium-operator ready: true, restart count 0 Jan 9 06:45:38.197: INFO: etcd-manager-main-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:38.197: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:45:38.197: INFO: kube-apiserver-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+2 container statuses recorded) Jan 9 06:45:38.197: INFO: Container healthcheck ready: true, restart count 0 Jan 9 06:45:38.197: INFO: Container kube-apiserver ready: true, restart count 1 Jan 9 06:45:38.197: INFO: cilium-97g4h started at 2023-01-09 06:24:28 +0000 UTC (1+1 container statuses recorded) Jan 9 06:45:38.197: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:45:38.197: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:45:38.197: INFO: aws-cloud-controller-manager-kj4b6 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:38.197: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 9 06:45:38.197: INFO: kops-controller-8qjsv started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:38.197: INFO: Container kops-controller ready: true, restart count 0 Jan 9 06:45:38.197: INFO: ebs-csi-node-kblmh started at 2023-01-09 06:24:28 +0000 UTC (0+3 container statuses recorded) Jan 9 06:45:38.197: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:45:38.197: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:45:38.197: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:45:38.197: INFO: ebs-csi-controller-89b649c94-zwk75 started at 2023-01-09 06:24:28 +0000 UTC (0+5 container statuses recorded) Jan 9 06:45:38.197: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:45:38.197: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:45:38.197: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:45:38.197: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:45:38.197: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:45:38.197: INFO: kube-controller-manager-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:38.197: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 9 06:45:38.197: INFO: kube-scheduler-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:38.197: INFO: Container kube-scheduler ready: true, restart count 0 Jan 9 06:45:38.197: INFO: etcd-manager-events-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:38.197: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:45:38.678: INFO: Latency metrics for node i-02b09d19b2096261b Jan 9 06:45:38.678: INFO: Logging node info for node i-0585809501bfc393d Jan 9 06:45:38.821: INFO: Node Info: &Node{ObjectMeta:{i-0585809501bfc393d 8c7c1c77-e673-43a0-81a4-fcf97493bcd8 29271 0 2023-01-09 06:25:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0585809501bfc393d kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-0585809501bfc393d topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.147 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0585809501bfc393d"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-09 06:41:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0585809501bfc393d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.147,},NodeAddress{Type:ExternalIP,Address:54.94.242.160,},NodeAddress{Type:InternalDNS,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-242-160.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec206fc38cf93b7ea12ba838ad958aa3,SystemUUID:ec206fc3-8cf9-3b7e-a12b-a838ad958aa3,BootID:afb66846-6976-49bf-9e61-b54e5af8a3eb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:45:38.821: INFO: Logging kubelet events for node i-0585809501bfc393d Jan 9 06:45:38.966: INFO: Logging pods the kubelet thinks is on node i-0585809501bfc393d Jan 9 06:45:39.117: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-cs2m5 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:39.117: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-97f28 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:39.117: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-p6pfg started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:39.117: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-6hzsb started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:39.117: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:40:25 +0000 UTC (0+7 container statuses recorded) Jan 9 06:45:39.117: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:45:39.117: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-bxrs7 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:39.117: INFO: netserver-0 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container webserver ready: true, restart count 0 Jan 9 06:45:39.117: INFO: cilium-f5j8w started at 2023-01-09 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:45:39.117: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:45:39.117: INFO: pod-secrets-2e68e54b-1a9c-4316-a873-c553608b50e7 started at 2023-01-09 06:44:42 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container secret-test ready: false, restart count 0 Jan 9 06:45:39.117: INFO: dns-test-bdcfcfaa-2367-4a35-bb69-c5dd9d18f66c started at 2023-01-09 06:44:29 +0000 UTC (0+3 container statuses recorded) Jan 9 06:45:39.117: INFO: Container jessie-querier ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container querier ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container webserver ready: false, restart count 0 Jan 9 06:45:39.117: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fs9d6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:39.117: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:39:17 +0000 UTC (0+7 container statuses recorded) Jan 9 06:45:39.117: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:45:39.117: INFO: ebs-csi-node-6djgn started at 2023-01-09 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 9 06:45:39.117: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:45:39.117: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:45:39.117: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:45:39.117: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fm99c started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:39.117: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-xtk85 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:39.117: INFO: alpine-nnp-false-4247b683-138d-4632-a6a3-6d844097c353 started at 2023-01-09 06:41:15 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container alpine-nnp-false-4247b683-138d-4632-a6a3-6d844097c353 ready: false, restart count 0 Jan 9 06:45:39.117: INFO: webserver-66b8c99f6-ls7f4 started at 2023-01-09 06:41:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container httpd ready: false, restart count 0 Jan 9 06:45:39.117: INFO: execpodjqtvg started at 2023-01-09 06:42:07 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container agnhost-container ready: false, restart count 0 Jan 9 06:45:39.117: INFO: busybox-scheduling-ab31e8f1-8d93-4600-a76d-d2581bb59688 started at 2023-01-09 06:44:53 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container busybox-scheduling-ab31e8f1-8d93-4600-a76d-d2581bb59688 ready: false, restart count 0 Jan 9 06:45:39.117: INFO: dns-test-a96d67de-eeb2-4e87-8d66-17c313a4ef5a started at 2023-01-09 06:40:06 +0000 UTC (0+3 container statuses recorded) Jan 9 06:45:39.117: INFO: Container jessie-querier ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container querier ready: false, restart count 0 Jan 9 06:45:39.117: INFO: Container webserver ready: false, restart count 0 Jan 9 06:45:39.117: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-qm6jm started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:39.117: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8rq57 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:39.117: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-2jgj6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:39.117: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:39.637: INFO: Latency metrics for node i-0585809501bfc393d Jan 9 06:45:39.637: INFO: Logging node info for node i-06d5855698aa7cb08 Jan 9 06:45:39.780: INFO: Node Info: &Node{ObjectMeta:{i-06d5855698aa7cb08 bdfd8f9d-5aa1-4cff-806a-849d1c29fcb6 31159 0 2023-01-09 06:25:34 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06d5855698aa7cb08 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-06d5855698aa7cb08 topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.46.163 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-06d5855698aa7cb08"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-09 06:45:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06d5855698aa7cb08,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:48 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:48 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:48 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:44:48 +0000 UTC,LastTransitionTime:2023-01-09 06:25:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.46.163,},NodeAddress{Type:ExternalIP,Address:18.228.235.107,},NodeAddress{Type:InternalDNS,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-107.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29e10a0890d67138c4eff5b197ee61,SystemUUID:ec29e10a-0890-d671-38c4-eff5b197ee61,BootID:9352599c-af91-4c75-860f-bb9a2789a998,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:45:39.781: INFO: Logging kubelet events for node i-06d5855698aa7cb08 Jan 9 06:45:39.926: INFO: Logging pods the kubelet thinks is on node i-06d5855698aa7cb08 Jan 9 06:45:40.086: INFO: webserver-66b8c99f6-t458w started at 2023-01-09 06:45:27 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:40.086: INFO: Container httpd ready: false, restart count 0 Jan 9 06:45:40.086: INFO: webserver-7f5969cbc7-tz85c started at 2023-01-09 06:41:16 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:40.086: INFO: Container httpd ready: true, restart count 0 Jan 9 06:45:40.086: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-vwlp2 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:40.086: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:40.086: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pnbg6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:40.086: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:40.086: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-mkn6f started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:40.086: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:40.086: INFO: netserver-1 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:40.086: INFO: Container webserver ready: false, restart count 0 Jan 9 06:45:40.086: INFO: ebs-csi-node-6dzh8 started at 2023-01-09 06:25:35 +0000 UTC (0+3 container statuses recorded) Jan 9 06:45:40.086: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:45:40.086: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:45:40.086: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:45:40.086: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:42:19 +0000 UTC (0+7 container statuses recorded) Jan 9 06:45:40.086: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:45:40.086: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:45:40.086: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:45:40.086: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:45:40.086: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:45:40.086: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:45:40.086: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:45:40.086: INFO: csi-mockplugin-0 started at 2023-01-09 06:40:46 +0000 UTC (0+3 container statuses recorded) Jan 9 06:45:40.086: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:45:40.086: INFO: Container driver-registrar ready: false, restart count 0 Jan 9 06:45:40.086: INFO: Container mock ready: false, restart count 0 Jan 9 06:45:40.086: INFO: coredns-559769c974-qdhtp started at 2023-01-09 06:26:21 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:40.086: INFO: Container coredns ready: true, restart count 0 Jan 9 06:45:40.086: INFO: csi-mockplugin-attacher-0 started at 2023-01-09 06:44:53 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:40.086: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:45:40.086: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8hxw6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:40.086: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:40.086: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-wg2gn started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:40.086: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:40.086: INFO: cilium-wbn8d started at 2023-01-09 06:25:35 +0000 UTC (1+1 container statuses recorded) Jan 9 06:45:40.086: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:45:40.086: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:45:40.086: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fncl4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:40.086: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:40.086: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kr2c4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:40.086: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:40.086: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-b42jp started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:40.086: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:40.086: INFO: csi-mockplugin-0 started at 2023-01-09 06:44:53 +0000 UTC (0+3 container statuses recorded) Jan 9 06:45:40.086: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:45:40.086: INFO: Container driver-registrar ready: false, restart count 0 Jan 9 06:45:40.086: INFO: Container mock ready: false, restart count 0 Jan 9 06:45:40.086: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-hpb4w started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:40.086: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:40.668: INFO: Latency metrics for node i-06d5855698aa7cb08 Jan 9 06:45:40.668: INFO: Logging node info for node i-087c2326a312024ef Jan 9 06:45:40.811: INFO: Node Info: &Node{ObjectMeta:{i-087c2326a312024ef 45db8cc8-250b-411a-b344-d6bbffe46b7d 29314 0 2023-01-09 06:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-087c2326a312024ef kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-087c2326a312024ef topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.218 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-087c2326a312024ef"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:40:12 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-09 06:41:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-087c2326a312024ef,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.218,},NodeAddress{Type:ExternalIP,Address:18.230.17.62,},NodeAddress{Type:InternalDNS,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-17-62.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec28fbdf9f29233021716214b9f8da3a,SystemUUID:ec28fbdf-9f29-2330-2171-6214b9f8da3a,BootID:4dbf8c75-8514-419f-b443-1a44f89f420f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d49342c84d2a787d],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d49342c84d2a787d,DevicePath:,},},Config:nil,},} Jan 9 06:45:40.811: INFO: Logging kubelet events for node i-087c2326a312024ef Jan 9 06:45:40.956: INFO: Logging pods the kubelet thinks is on node i-087c2326a312024ef Jan 9 06:45:41.106: INFO: pod-1f33e7e4-d6e6-49be-b970-1717d19d4b2c started at 2023-01-09 06:44:59 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.106: INFO: Container write-pod ready: false, restart count 0 Jan 9 06:45:41.106: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-qxtwd started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.106: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:41.106: INFO: pod-8a9d115a-40b6-4299-8082-2ec7acf6e83d started at 2023-01-09 06:39:05 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.106: INFO: Container write-pod ready: true, restart count 0 Jan 9 06:45:41.106: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-b8q64 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.106: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:41.106: INFO: test-rs-gj54c started at 2023-01-09 06:44:23 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.106: INFO: Container httpd ready: false, restart count 0 Jan 9 06:45:41.106: INFO: webserver-66b8c99f6-gt295 started at 2023-01-09 06:41:30 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.106: INFO: Container httpd ready: true, restart count 0 Jan 9 06:45:41.106: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-85kn4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.106: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:41.106: INFO: sample-webhook-deployment-865554f4d9-nll6q started at 2023-01-09 06:44:56 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.106: INFO: Container sample-webhook ready: false, restart count 0 Jan 9 06:45:41.106: INFO: var-expansion-a2239f26-e74b-4412-8ed1-edb727aba048 started at 2023-01-09 06:44:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.106: INFO: Container dapi-container ready: false, restart count 0 Jan 9 06:45:41.106: INFO: pod1 started at 2023-01-09 06:40:38 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.106: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:45:41.106: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-dlhg8 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.106: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:45:41.106: INFO: sample-webhook-deployment-865554f4d9-sqdbc started at 2023-01-09 06:44:13 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.106: INFO: Container sample-webhook ready: false, restart count 0 Jan 9 06:45:41.106: INFO: netserver-2 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.106: INFO: Container webserver ready: true, restart count 0 Jan 9 06:45:41.107: INFO: downwardapi-volume-b6e1bef1-7e68-4e1f-b89e-adea2f3aafd9 started at 2023-01-09 06:44:55 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.107: INFO: Container client-container ready: false, restart count 0 Jan 9 06:45:41.107: INFO: ebs-csi-node-jvd2b started at 2023-01-09 06:25:35 +0000 UTC (0+3 container statuses recorded) Jan 9 06:45:41.107: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:45:41.107: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:45:41.107: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:45:41.107: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-h8zj6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.107: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:41.107: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pvc5d started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.107: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:45:41.107: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-r89ft started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.107: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:41.107: INFO: cilium-gw6mm started at 2023-01-09 06:25:35 +0000 UTC (1+1 container statuses recorded) Jan 9 06:45:41.107: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:45:41.107: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:45:41.107: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-plbnr started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.107: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:41.107: INFO: hostexec-i-087c2326a312024ef-srclt started at 2023-01-09 06:44:43 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.107: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:45:41.107: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pcnbb started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.107: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:41.107: INFO: pod-should-be-evicted95f3032f-4412-4cf0-898f-db2e0558e909 started at 2023-01-09 06:44:26 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.107: INFO: Container bar ready: false, restart count 0 Jan 9 06:45:41.107: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8brjv started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:41.107: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:41.628: INFO: Latency metrics for node i-087c2326a312024ef Jan 9 06:45:41.628: INFO: Logging node info for node i-0c0b18eb32e7da83d Jan 9 06:45:41.770: INFO: Node Info: &Node{ObjectMeta:{i-0c0b18eb32e7da83d f1550edf-43cd-451c-bf01-eb92d6980d45 30343 0 2023-01-09 06:25:33 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0c0b18eb32e7da83d kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-0c0b18eb32e7da83d topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.55.27 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0c0b18eb32e7da83d"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-09 06:44:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0c0b18eb32e7da83d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:33 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:44:06 +0000 UTC,LastTransitionTime:2023-01-09 06:25:51 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.55.27,},NodeAddress{Type:ExternalIP,Address:18.228.15.186,},NodeAddress{Type:InternalDNS,Address:i-0c0b18eb32e7da83d.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0c0b18eb32e7da83d.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-15-186.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec267e00a4086801bd60f1191c81f587,SystemUUID:ec267e00-a408-6801-bd60-f1191c81f587,BootID:7c624b34-f575-4c40-8226-aef6612a1a56,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:45:41.771: INFO: Logging kubelet events for node i-0c0b18eb32e7da83d Jan 9 06:45:41.915: INFO: Logging pods the kubelet thinks is on node i-0c0b18eb32e7da83d Jan 9 06:45:42.069: INFO: hostexec-i-0c0b18eb32e7da83d-sj9mw started at 2023-01-09 06:45:26 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:45:42.069: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kpwrs started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: false, restart count 0 Jan 9 06:45:42.069: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kxqfw started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:42.069: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-ck9r4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:42.069: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-lknfk started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:42.069: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-cfps7 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:42.069: INFO: netserver-3 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container webserver ready: false, restart count 0 Jan 9 06:45:42.069: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-njsp9 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:42.069: INFO: webserver-66b8c99f6-b9md9 started at 2023-01-09 06:44:35 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container httpd ready: true, restart count 0 Jan 9 06:45:42.069: INFO: ebs-csi-node-m5kg7 started at 2023-01-09 06:25:34 +0000 UTC (0+3 container statuses recorded) Jan 9 06:45:42.069: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:45:42.069: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:45:42.069: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:45:42.069: INFO: cilium-vx7vq started at 2023-01-09 06:25:34 +0000 UTC (1+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:45:42.069: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:45:42.069: INFO: coredns-559769c974-md8gz started at 2023-01-09 06:25:51 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container coredns ready: true, restart count 0 Jan 9 06:45:42.069: INFO: coredns-autoscaler-7cb5c5b969-4vwwx started at 2023-01-09 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container autoscaler ready: true, restart count 0 Jan 9 06:45:42.069: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-sf7zm started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:42.069: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-mwm9d started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:42.069: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-twg96 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:42.069: INFO: test-rs-xfx6b started at 2023-01-09 06:45:38 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container httpd ready: false, restart count 0 Jan 9 06:45:42.069: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-ft8kr started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:45:42.069: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:45:42.595: INFO: Latency metrics for node i-0c0b18eb32e7da83d [DeferCleanup (Each)] [sig-network] Networking tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "nettest-1377" for this suite. �[38;5;243m01/09/23 06:45:42.596�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-node\]\sContainer\sRuntime\sblackbox\stest\son\sterminated\scontainer\sshould\sreport\stermination\smessage\sfrom\slog\soutput\sif\sTerminationMessagePolicy\sFallbackToLogsOnError\sis\sset\s\[NodeConformance\]\s\[Conformance\]$'
test/e2e/common/node/runtime.go:157 k8s.io/kubernetes/test/e2e/common/node.glob..func18.1.2.1({{0x764326c, 0x1d}, {0xc000212c00, 0x2e}, {0xc00215a3e0, 0x2, 0x2}, {0xc0012ac040, 0x1, 0x1}, ...}, ...) test/e2e/common/node/runtime.go:157 +0x38c k8s.io/kubernetes/test/e2e/common/node.glob..func18.1.2.4() test/e2e/common/node/runtime.go:224 +0x16e
[BeforeEach] [sig-node] Container Runtime set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/09/23 06:39:18.822�[0m Jan 9 06:39:18.822: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename container-runtime �[38;5;243m01/09/23 06:39:18.823�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/09/23 06:39:19.273�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/09/23 06:39:19.555�[0m [BeforeEach] [sig-node] Container Runtime test/e2e/framework/metrics/init/init.go:31 [It] should report termination message from log output if TerminationMessagePolicy FallbackToLogsOnError is set [NodeConformance] [Conformance] test/e2e/common/node/runtime.go:216 �[1mSTEP:�[0m create the container �[38;5;243m01/09/23 06:39:19.838�[0m �[1mSTEP:�[0m wait for the container to reach Failed �[38;5;243m01/09/23 06:39:19.983�[0m Jan 9 06:44:19.984: FAIL: Timed out after 300.001s. Expected <v1.PodPhase>: Pending to equal <v1.PodPhase>: Failed Full Stack Trace k8s.io/kubernetes/test/e2e/common/node.glob..func18.1.2.1({{0x764326c, 0x1d}, {0xc000212c00, 0x2e}, {0xc00215a3e0, 0x2, 0x2}, {0xc0012ac040, 0x1, 0x1}, ...}, ...) test/e2e/common/node/runtime.go:157 +0x38c k8s.io/kubernetes/test/e2e/common/node.glob..func18.1.2.4() test/e2e/common/node/runtime.go:224 +0x16e [AfterEach] [sig-node] Container Runtime test/e2e/framework/node/init/init.go:32 Jan 9 06:44:20.131: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-node] Container Runtime test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-node] Container Runtime dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/09/23 06:44:20.275�[0m �[1mSTEP:�[0m Collecting events from namespace "container-runtime-3080". �[38;5;243m01/09/23 06:44:20.275�[0m �[1mSTEP:�[0m Found 11 events. �[38;5;243m01/09/23 06:44:20.418�[0m Jan 9 06:44:20.418: INFO: At 2023-01-09 06:39:19 +0000 UTC - event for termination-message-containercb256c75-c1c4-4b0e-b8ef-fa140a70f4ce: {default-scheduler } Scheduled: Successfully assigned container-runtime-3080/termination-message-containercb256c75-c1c4-4b0e-b8ef-fa140a70f4ce to i-087c2326a312024ef Jan 9 06:44:20.418: INFO: At 2023-01-09 06:39:20 +0000 UTC - event for termination-message-containercb256c75-c1c4-4b0e-b8ef-fa140a70f4ce: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "de2e255c0a2df0289e3d0609f1568b42fb952dabd2865be38f27beda00f0d29b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:20.418: INFO: At 2023-01-09 06:39:36 +0000 UTC - event for termination-message-containercb256c75-c1c4-4b0e-b8ef-fa140a70f4ce: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "be88cdd5b62394ea8ee29b83e0505dc2bbb5f5e08d8f7758cf3a69fd4e3905c2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:20.418: INFO: At 2023-01-09 06:39:50 +0000 UTC - event for termination-message-containercb256c75-c1c4-4b0e-b8ef-fa140a70f4ce: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ea7cdd60571e9becfd6ded9ce5815752a97903ec0be0009dca696b0f1224672e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:20.418: INFO: At 2023-01-09 06:40:01 +0000 UTC - event for termination-message-containercb256c75-c1c4-4b0e-b8ef-fa140a70f4ce: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cba26a3c6ba1edf93a739e354069efb6d00def59151be0ce443f90dfebbd79f4": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:20.418: INFO: At 2023-01-09 06:40:12 +0000 UTC - event for termination-message-containercb256c75-c1c4-4b0e-b8ef-fa140a70f4ce: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d3d74fa1332a5915f55558706066da5fb02f0ab46b15071cea76abbe6a8d270d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:20.418: INFO: At 2023-01-09 06:40:23 +0000 UTC - event for termination-message-containercb256c75-c1c4-4b0e-b8ef-fa140a70f4ce: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "2d376982969df6b06fbeb480f8ece9a00136f2081168e1e9bfd4e91d0e381658": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:20.418: INFO: At 2023-01-09 06:40:36 +0000 UTC - event for termination-message-containercb256c75-c1c4-4b0e-b8ef-fa140a70f4ce: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "47fa9ca4eaec6ddb30903650e6119c4050dd42208f6f46007d4f942a812bf71e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:20.418: INFO: At 2023-01-09 06:40:47 +0000 UTC - event for termination-message-containercb256c75-c1c4-4b0e-b8ef-fa140a70f4ce: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "acb599179671b125731ff9a673574e3d539f5f58f821e08c4acdc54aff12271b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:20.418: INFO: At 2023-01-09 06:40:59 +0000 UTC - event for termination-message-containercb256c75-c1c4-4b0e-b8ef-fa140a70f4ce: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "49653d55493dcfbdce723fd513b084003464ddfcccadaaf986255052c72c8318": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:20.418: INFO: At 2023-01-09 06:41:13 +0000 UTC - event for termination-message-containercb256c75-c1c4-4b0e-b8ef-fa140a70f4ce: {kubelet i-087c2326a312024ef} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c0137d7038c3f016cb28a6ebb0afd013e5a143dc96614d268e63f1f7d42057c6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 9 06:44:20.559: INFO: POD NODE PHASE GRACE CONDITIONS Jan 9 06:44:20.559: INFO: Jan 9 06:44:20.702: INFO: Logging node info for node i-02b09d19b2096261b Jan 9 06:44:20.844: INFO: Node Info: &Node{ObjectMeta:{i-02b09d19b2096261b efa11f02-6d54-4902-bbeb-efecc1f02cd1 29640 0 2023-01-09 06:23:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-02b09d19b2096261b kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:sa-east-1a topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.12 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-02b09d19b2096261b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-09 06:23:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-09 06:24:22 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:24:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-09 06:24:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-09 06:41:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-02b09d19b2096261b,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:23:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:37 +0000 UTC,LastTransitionTime:2023-01-09 06:24:43 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.12,},NodeAddress{Type:ExternalIP,Address:54.94.119.205,},NodeAddress{Type:InternalDNS,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-02b09d19b2096261b.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-119-205.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2da51bd0e40b36975082ac186a369f,SystemUUID:ec2da51b-d0e4-0b36-9750-82ac186a369f,BootID:83891f80-d7aa-4da9-ab10-44706ab3b631,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43453930,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42803458,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967347,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:20.844: INFO: Logging kubelet events for node i-02b09d19b2096261b Jan 9 06:44:20.988: INFO: Logging pods the kubelet thinks is on node i-02b09d19b2096261b Jan 9 06:44:21.135: INFO: kube-controller-manager-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.135: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 9 06:44:21.135: INFO: kube-scheduler-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.135: INFO: Container kube-scheduler ready: true, restart count 0 Jan 9 06:44:21.135: INFO: etcd-manager-events-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.135: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:44:21.135: INFO: aws-cloud-controller-manager-kj4b6 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.135: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 9 06:44:21.135: INFO: kops-controller-8qjsv started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.135: INFO: Container kops-controller ready: true, restart count 0 Jan 9 06:44:21.135: INFO: ebs-csi-node-kblmh started at 2023-01-09 06:24:28 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:21.135: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:21.135: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:21.135: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:21.135: INFO: ebs-csi-controller-89b649c94-zwk75 started at 2023-01-09 06:24:28 +0000 UTC (0+5 container statuses recorded) Jan 9 06:44:21.135: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:44:21.135: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:44:21.135: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:44:21.135: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:21.135: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:21.135: INFO: etcd-manager-main-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.135: INFO: Container etcd-manager ready: true, restart count 0 Jan 9 06:44:21.135: INFO: kube-apiserver-i-02b09d19b2096261b started at 2023-01-09 06:23:18 +0000 UTC (0+2 container statuses recorded) Jan 9 06:44:21.135: INFO: Container healthcheck ready: true, restart count 0 Jan 9 06:44:21.135: INFO: Container kube-apiserver ready: true, restart count 1 Jan 9 06:44:21.135: INFO: cilium-97g4h started at 2023-01-09 06:24:28 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:21.135: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:21.135: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:21.135: INFO: dns-controller-69987775c6-mv7h2 started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.135: INFO: Container dns-controller ready: true, restart count 0 Jan 9 06:44:21.135: INFO: cilium-operator-5dd44dc49f-fgc7c started at 2023-01-09 06:24:28 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:21.135: INFO: Container cilium-operator ready: true, restart count 0 Jan 9 06:44:21.619: INFO: Latency metrics for node i-02b09d19b2096261b Jan 9 06:44:21.619: INFO: Logging node info for node i-0585809501bfc393d Jan 9 06:44:21.761: INFO: Node Info: &Node{ObjectMeta:{i-0585809501bfc393d 8c7c1c77-e673-43a0-81a4-fcf97493bcd8 29271 0 2023-01-09 06:25:36 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0585809501bfc393d kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-0585809501bfc393d topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.147 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0585809501bfc393d"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.5.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-09 06:41:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.5.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-0585809501bfc393d,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.5.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:36 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:04 +0000 UTC,LastTransitionTime:2023-01-09 06:25:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.147,},NodeAddress{Type:ExternalIP,Address:54.94.242.160,},NodeAddress{Type:InternalDNS,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0585809501bfc393d.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-94-242-160.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec206fc38cf93b7ea12ba838ad958aa3,SystemUUID:ec206fc3-8cf9-3b7e-a12b-a838ad958aa3,BootID:afb66846-6976-49bf-9e61-b54e5af8a3eb,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 9 06:44:21.761: INFO: Logging kubelet events for node i-0585809501bfc393d Jan 9 06:44:21.905: INFO: Logging pods the kubelet thinks is on node i-0585809501bfc393d Jan 9 06:44:22.056: INFO: netserver-0 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:22.056: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-bxrs7 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.056: INFO: netserver-0 started at 2023-01-09 06:39:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:22.056: INFO: cilium-f5j8w started at 2023-01-09 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:22.056: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:22.056: INFO: webserver-7f5969cbc7-bm97p started at 2023-01-09 06:41:26 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:22.056: INFO: alpine-nnp-true-7168054a-3362-4368-bc5e-54ecf20fc5ca started at 2023-01-09 06:39:32 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container alpine-nnp-true-7168054a-3362-4368-bc5e-54ecf20fc5ca ready: false, restart count 0 Jan 9 06:44:22.056: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:39:17 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:22.056: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:22.056: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fs9d6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.056: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fm99c started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.056: INFO: ebs-csi-node-6djgn started at 2023-01-09 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:22.056: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:22.056: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:22.056: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:22.056: INFO: alpine-nnp-false-4247b683-138d-4632-a6a3-6d844097c353 started at 2023-01-09 06:41:15 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container alpine-nnp-false-4247b683-138d-4632-a6a3-6d844097c353 ready: false, restart count 0 Jan 9 06:44:22.056: INFO: webserver-66b8c99f6-ls7f4 started at 2023-01-09 06:41:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:22.056: INFO: execpodjqtvg started at 2023-01-09 06:42:07 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container agnhost-container ready: false, restart count 0 Jan 9 06:44:22.056: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-xtk85 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.056: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-qm6jm started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.056: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8rq57 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.056: INFO: update-demo-nautilus-gqbgn started at 2023-01-09 06:39:17 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container update-demo ready: false, restart count 0 Jan 9 06:44:22.056: INFO: dns-test-a96d67de-eeb2-4e87-8d66-17c313a4ef5a started at 2023-01-09 06:40:06 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:22.056: INFO: Container jessie-querier ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Container querier ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:22.056: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-2jgj6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.056: INFO: pod-subpath-test-preprovisionedpv-wp87 started at 2023-01-09 06:39:43 +0000 UTC (2+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Init container init-volume-preprovisionedpv-wp87 ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Init container test-init-subpath-preprovisionedpv-wp87 ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Container test-container-subpath-preprovisionedpv-wp87 ready: false, restart count 0 Jan 9 06:44:22.056: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-cs2m5 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.056: INFO: test-rollover-controller-75rk6 started at 2023-01-09 06:39:32 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:22.056: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-97f28 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.056: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-p6pfg started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.056: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-6hzsb started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:22.056: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:40:25 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:22.056: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:22.056: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:22.056: INFO: hostexec-i-0585809501bfc393d-v4gr6 started at 2023-01-09 06:39:24 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:22.056: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:22.644: INFO: Latency metrics for node i-0585809501bfc393d Jan 9 06:44:22.644: INFO: Logging node info for node i-06d5855698aa7cb08 Jan 9 06:44:22.786: INFO: Node Info: &Node{ObjectMeta:{i-06d5855698aa7cb08 bdfd8f9d-5aa1-4cff-806a-849d1c29fcb6 28608 0 2023-01-09 06:25:34 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-06d5855698aa7cb08 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-06d5855698aa7cb08 topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.46.163 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-2682":"i-06d5855698aa7cb08","ebs.csi.aws.com":"i-06d5855698aa7cb08"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:39:47 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-09 06:39:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-06d5855698aa7cb08,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:39:52 +0000 UTC,LastTransitionTime:2023-01-09 06:25:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.46.163,},NodeAddress{Type:ExternalIP,Address:18.228.235.107,},NodeAddress{Type:InternalDNS,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-06d5855698aa7cb08.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-228-235-107.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec29e10a0890d67138c4eff5b197ee61,SystemUUID:ec29e10a-0890-d671-38c4-eff5b197ee61,BootID:9352599c-af91-4c75-860f-bb9a2789a998,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-2682^66d24229-8fe8-11ed-abbf-d2969ce11d4b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-2682^66d24229-8fe8-11ed-abbf-d2969ce11d4b,DevicePath:,},},Config:nil,},} Jan 9 06:44:22.786: INFO: Logging kubelet events for node i-06d5855698aa7cb08 Jan 9 06:44:22.930: INFO: Logging pods the kubelet thinks is on node i-06d5855698aa7cb08 Jan 9 06:44:23.080: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:39:00 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:23.080: INFO: Container csi-attacher ready: true, restart count 0 Jan 9 06:44:23.080: INFO: Container csi-provisioner ready: true, restart count 0 Jan 9 06:44:23.080: INFO: Container csi-resizer ready: true, restart count 0 Jan 9 06:44:23.080: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 9 06:44:23.080: INFO: Container hostpath ready: true, restart count 0 Jan 9 06:44:23.080: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:23.080: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:23.080: INFO: csi-hostpathplugin-0 started at 2023-01-09 06:42:19 +0000 UTC (0+7 container statuses recorded) Jan 9 06:44:23.080: INFO: Container csi-attacher ready: false, restart count 0 Jan 9 06:44:23.080: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:23.080: INFO: Container csi-resizer ready: false, restart count 0 Jan 9 06:44:23.080: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 9 06:44:23.080: INFO: Container hostpath ready: false, restart count 0 Jan 9 06:44:23.080: INFO: Container liveness-probe ready: false, restart count 0 Jan 9 06:44:23.080: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 9 06:44:23.080: INFO: coredns-559769c974-qdhtp started at 2023-01-09 06:26:21 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container coredns ready: true, restart count 0 Jan 9 06:44:23.080: INFO: netserver-1 started at 2023-01-09 06:39:18 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container webserver ready: true, restart count 0 Jan 9 06:44:23.080: INFO: csi-mockplugin-0 started at 2023-01-09 06:40:46 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:23.080: INFO: Container csi-provisioner ready: false, restart count 0 Jan 9 06:44:23.080: INFO: Container driver-registrar ready: false, restart count 0 Jan 9 06:44:23.080: INFO: Container mock ready: false, restart count 0 Jan 9 06:44:23.080: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-wg2gn started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.080: INFO: pod-d7eb61ad-d823-42ce-b057-d1397e0091be started at 2023-01-09 06:39:46 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container write-pod ready: false, restart count 0 Jan 9 06:44:23.080: INFO: cilium-wbn8d started at 2023-01-09 06:25:35 +0000 UTC (1+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 9 06:44:23.080: INFO: Container cilium-agent ready: true, restart count 0 Jan 9 06:44:23.080: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-fncl4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.080: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-8hxw6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.080: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-b42jp started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.080: INFO: hostexec-i-06d5855698aa7cb08-gdgwr started at 2023-01-09 06:39:24 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container agnhost-container ready: true, restart count 0 Jan 9 06:44:23.080: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-hpb4w started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.080: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-kr2c4 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.080: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-vwlp2 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.080: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-pnbg6 started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.080: INFO: cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5-mkn6f started at 2023-01-09 06:39:00 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container cleanup40-40bec070-581a-46a4-ab91-92caa48dc6e5 ready: true, restart count 0 Jan 9 06:44:23.080: INFO: pod-fe7ddf40-2506-41f1-af53-09d0de5705e7 started at 2023-01-09 06:39:29 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container write-pod ready: false, restart count 0 Jan 9 06:44:23.080: INFO: webserver-7f5969cbc7-tz85c started at 2023-01-09 06:41:16 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container httpd ready: false, restart count 0 Jan 9 06:44:23.080: INFO: ebs-csi-node-6dzh8 started at 2023-01-09 06:25:35 +0000 UTC (0+3 container statuses recorded) Jan 9 06:44:23.080: INFO: Container ebs-plugin ready: true, restart count 0 Jan 9 06:44:23.080: INFO: Container liveness-probe ready: true, restart count 0 Jan 9 06:44:23.080: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 9 06:44:23.080: INFO: netserver-1 started at 2023-01-09 06:39:33 +0000 UTC (0+1 container statuses recorded) Jan 9 06:44:23.080: INFO: Container webserver ready: false, restart count 0 Jan 9 06:44:23.632: INFO: Latency metrics for node i-06d5855698aa7cb08 Jan 9 06:44:23.632: INFO: Logging node info for node i-087c2326a312024ef Jan 9 06:44:23.774: INFO: Node Info: &Node{ObjectMeta:{i-087c2326a312024ef 45db8cc8-250b-411a-b344-d6bbffe46b7d 29314 0 2023-01-09 06:25:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:sa-east-1 failure-domain.beta.kubernetes.io/zone:sa-east-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-087c2326a312024ef kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:sa-east-1a topology.hostpath.csi/node:i-087c2326a312024ef topology.kubernetes.io/region:sa-east-1 topology.kubernetes.io/zone:sa-east-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.218 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-087c2326a312024ef"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-09 06:25:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-09 06:38:59 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-09 06:40:12 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-09 06:41:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///sa-east-1a/i-087c2326a312024ef,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051689472 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946831872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:34 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-09 06:41:13 +0000 UTC,LastTransitionTime:2023-01-09 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.218,},NodeAddress{Type:ExternalIP,Address:18.230.17.62,},NodeAddress{Type:InternalDNS,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-087c2326a312024ef.sa-east-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-18-230-17-62.sa-east-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec28fbdf9f29233021716214b9f8da3a,SystemUUID:ec28fbdf-9f29-2330-2171-6214b9f8da3a,BootID:4dbf8c75-8514-419f-b443-1a44f89f420f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImag