e2e-kops-grid-cilium-eni-u2004-k26 #1618131493396156416
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 7 failed / 879 succeeded |
| Started | |
| Elapsed | 40m12s |
| Revision | master |
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\]\svolumeMode\sshould\snot\smount\s\/\smap\sunused\svolumes\sin\sa\spod\s\[LinuxOnly\]$'
test/e2e/storage/testsuites/volumemode.go:387 k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeModeTestSuite).DefineTests.func7() test/e2e/storage/testsuites/volumemode.go:387 +0x62bfrom junit_01.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/25/23 06:47:43.175�[0m Jan 25 06:47:43.175: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename volumemode �[38;5;243m01/25/23 06:47:43.176�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/25/23 06:47:43.271�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/25/23 06:47:43.332�[0m [BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode test/e2e/framework/metrics/init/init.go:31 [It] should not mount / map unused volumes in a pod [LinuxOnly] test/e2e/storage/testsuites/volumemode.go:354 Jan 25 06:47:43.393: INFO: Creating resource for dynamic PV Jan 25 06:47:43.393: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} �[1mSTEP:�[0m creating a StorageClass volumemode-9184-e2e-scq46df �[38;5;243m01/25/23 06:47:43.393�[0m �[1mSTEP:�[0m creating a claim �[38;5;243m01/25/23 06:47:43.426�[0m �[1mSTEP:�[0m Creating pod �[38;5;243m01/25/23 06:47:43.496�[0m Jan 25 06:47:43.529: INFO: Waiting up to 5m0s for pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62" in namespace "volumemode-9184" to be "running" Jan 25 06:47:43.560: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 31.415081ms Jan 25 06:47:45.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2.063720706s Jan 25 06:47:47.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4.063491345s Jan 25 06:47:49.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 6.063528178s Jan 25 06:47:51.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 8.06365027s Jan 25 06:47:53.594: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 10.065676634s Jan 25 06:47:55.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 12.063098971s Jan 25 06:47:57.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 14.063428132s Jan 25 06:47:59.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 16.06345415s Jan 25 06:48:01.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 18.063073997s Jan 25 06:48:03.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 20.063596761s Jan 25 06:48:05.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 22.064145035s Jan 25 06:48:07.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 24.06411429s Jan 25 06:48:09.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 26.063551325s Jan 25 06:48:11.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 28.063685559s Jan 25 06:48:13.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 30.063671198s Jan 25 06:48:15.594: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 32.065664404s Jan 25 06:48:17.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 34.063518655s Jan 25 06:48:19.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 36.064861794s Jan 25 06:48:21.595: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 38.06664031s Jan 25 06:48:23.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 40.063259989s Jan 25 06:48:25.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 42.064274355s Jan 25 06:48:27.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 44.063082857s Jan 25 06:48:29.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 46.063946007s Jan 25 06:48:31.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 48.063624635s Jan 25 06:48:33.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 50.063748459s Jan 25 06:48:35.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 52.063103485s Jan 25 06:48:37.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 54.06381823s Jan 25 06:48:39.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 56.063674657s Jan 25 06:48:41.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 58.063056825s Jan 25 06:48:43.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.064240053s Jan 25 06:48:45.591: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.062967424s Jan 25 06:48:47.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.063762314s Jan 25 06:48:49.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.063563342s Jan 25 06:48:51.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.063628595s Jan 25 06:48:53.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.063737514s Jan 25 06:48:55.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.063689926s Jan 25 06:48:57.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.063686529s Jan 25 06:48:59.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.063674204s Jan 25 06:49:01.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.06370981s Jan 25 06:49:03.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.063030476s Jan 25 06:49:05.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.063964204s Jan 25 06:49:07.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.063571937s Jan 25 06:49:09.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.063755979s Jan 25 06:49:11.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.063843016s Jan 25 06:49:13.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.063424156s Jan 25 06:49:15.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.063495209s Jan 25 06:49:17.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.063766367s Jan 25 06:49:19.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.06362877s Jan 25 06:49:21.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.063537762s Jan 25 06:49:23.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.063179759s Jan 25 06:49:25.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.0646754s Jan 25 06:49:27.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.063534903s Jan 25 06:49:29.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.06377716s Jan 25 06:49:31.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.063339236s Jan 25 06:49:33.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.063598589s Jan 25 06:49:35.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.064637807s Jan 25 06:49:37.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.063221605s Jan 25 06:49:39.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.063850695s Jan 25 06:49:41.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.063704637s Jan 25 06:49:43.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.063656821s Jan 25 06:49:45.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.062996153s Jan 25 06:49:47.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.06392681s Jan 25 06:49:49.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.063224662s Jan 25 06:49:51.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.064104933s Jan 25 06:49:53.594: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.065349854s Jan 25 06:49:55.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.063833216s Jan 25 06:49:57.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.063392431s Jan 25 06:49:59.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.063442371s Jan 25 06:50:01.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.063825026s Jan 25 06:50:03.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.063468924s Jan 25 06:50:05.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.063897672s Jan 25 06:50:07.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.063803756s Jan 25 06:50:09.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.063651597s Jan 25 06:50:11.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.063379755s Jan 25 06:50:13.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.064186167s Jan 25 06:50:15.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.063274701s Jan 25 06:50:17.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.063080602s Jan 25 06:50:19.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.063312755s Jan 25 06:50:21.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.063913043s Jan 25 06:50:23.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.063688846s Jan 25 06:50:25.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.0632956s Jan 25 06:50:27.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.063175955s Jan 25 06:50:29.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.0639175s Jan 25 06:50:31.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.063688015s Jan 25 06:50:33.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.063096691s Jan 25 06:50:35.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.063113018s Jan 25 06:50:37.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.063609619s Jan 25 06:50:39.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.063614789s Jan 25 06:50:41.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.063630243s Jan 25 06:50:43.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.063520298s Jan 25 06:50:45.591: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.062921206s Jan 25 06:50:47.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.063002655s Jan 25 06:50:49.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.063276397s Jan 25 06:50:51.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.063764422s Jan 25 06:50:53.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.063699098s Jan 25 06:50:55.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.063428378s Jan 25 06:50:57.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.063126433s Jan 25 06:50:59.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.063638497s Jan 25 06:51:01.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.0638944s Jan 25 06:51:03.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.06352617s Jan 25 06:51:05.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.063662294s Jan 25 06:51:07.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.063473075s Jan 25 06:51:09.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.063500204s Jan 25 06:51:11.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.063247462s Jan 25 06:51:13.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.064121375s Jan 25 06:51:15.594: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.065720213s Jan 25 06:51:17.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.063608672s Jan 25 06:51:19.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.063849467s Jan 25 06:51:21.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.063793028s Jan 25 06:51:23.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.063625359s Jan 25 06:51:25.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.063371481s Jan 25 06:51:27.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.064183378s Jan 25 06:51:29.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.063199244s Jan 25 06:51:31.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.063323166s Jan 25 06:51:33.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.063354447s Jan 25 06:51:35.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.063711439s Jan 25 06:51:37.602: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.073217029s Jan 25 06:51:39.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.063596746s Jan 25 06:51:41.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.063292411s Jan 25 06:51:43.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.064070961s Jan 25 06:51:45.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.063308739s Jan 25 06:51:47.591: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.062815418s Jan 25 06:51:49.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.06372229s Jan 25 06:51:51.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.06361659s Jan 25 06:51:53.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.063319673s Jan 25 06:51:55.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.063617849s Jan 25 06:51:57.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.063925705s Jan 25 06:51:59.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.064421068s Jan 25 06:52:01.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.063779331s Jan 25 06:52:03.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.063848824s Jan 25 06:52:05.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.063760439s Jan 25 06:52:07.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.064172013s Jan 25 06:52:09.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.064057239s Jan 25 06:52:11.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.06382572s Jan 25 06:52:13.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.063661583s Jan 25 06:52:15.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.063622762s Jan 25 06:52:17.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.063510125s Jan 25 06:52:19.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.063654067s Jan 25 06:52:21.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.063244154s Jan 25 06:52:23.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.063692607s Jan 25 06:52:25.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.063767473s Jan 25 06:52:27.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.063631226s Jan 25 06:52:29.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.06372907s Jan 25 06:52:31.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.063720733s Jan 25 06:52:33.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.063363443s Jan 25 06:52:35.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.063372862s Jan 25 06:52:37.592: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.06320294s Jan 25 06:52:39.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.063976783s Jan 25 06:52:41.593: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.064074077s Jan 25 06:52:43.594: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.065306815s Jan 25 06:52:43.625: INFO: Pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.096684179s Jan 25 06:52:43.627: INFO: Unexpected error: <*pod.timeoutError | 0xc002d97320>: { msg: "timed out while waiting for pod volumemode-9184/pod-c185a4fa-326c-4f33-957a-bddf24584c62 to be running", observedObjects: [ <*v1.Pod | 0xc001683b00>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "pod-c185a4fa-326c-4f33-957a-bddf24584c62", GenerateName: "", Namespace: "volumemode-9184", SelfLink: "", UID: "f5cf87a1-48c1-4939-8a7c-06e045ae822e", ResourceVersion: "45166", Generation: 0, CreationTimestamp: { Time: { wall: 0, ext: 63810226063, loc: { name: "Local", zone: [ {name: "UTC", offset: 0, isDST: false}, ], tx: [ { when: -576460752303423488, index: 0, isstd: false, isutc: false, }, ], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: "UTC", offset: 0, isDST: false}, }, }, }, DeletionTimestamp: nil, DeletionGracePeriodSeconds: nil, Labels: nil, Annotations: nil, OwnerReferences: nil, Finalizers: nil, ManagedFields: [ { Manager: "e2e.test", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63810226063, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"write-pod\\\"}\":{\".\":{},\"f:command\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:securityContext\":{\".\":{},\"f:privileged\":{}},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{\".\":{},\"f:fsGroup\":{},\"f:seLinuxOptions\":{\".\":{},\"f:level\":{}}},\"f:terminationGracePeriodSeconds\":{},\"f:volumes\":{\".\":{},\"k:{\\\"name\\\":\\\"volume1\\\"}\":{\".\":{},\"f:name\":{},\"f:persistentVolumeClaim\":{\".\":{},\"f:claimName\":{}}}}}}", }, Subresource: "", }, { Manager: "kubelet", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63810226067, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}", }, Subresource: "status", }, ], }, Spec: { Volumes: [ { Name: "volume1", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: { ClaimName: "ebs.csi.aws.comrd5b5", ReadOnly: false, }, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: nil, PortworxVolume: nil, ScaleIO: nil, StorageOS: nil, CSI: nil, Ephemeral: nil, }, }, { Name: "kube-api-access-bfgkv", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Jan 25 06:52:43.627: FAIL: timed out while waiting for pod volumemode-9184/pod-c185a4fa-326c-4f33-957a-bddf24584c62 to be running Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumeModeTestSuite).DefineTests.func7() test/e2e/storage/testsuites/volumemode.go:387 +0x62b Jan 25 06:52:43.628: INFO: Deleting pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62" in namespace "volumemode-9184" Jan 25 06:52:43.662: INFO: Wait up to 5m0s for pod "pod-c185a4fa-326c-4f33-957a-bddf24584c62" to be fully deleted �[1mSTEP:�[0m Deleting pvc �[38;5;243m01/25/23 06:52:45.728�[0m Jan 25 06:52:45.790: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comrd5b5" Jan 25 06:52:45.822: INFO: Waiting up to 5m0s for PersistentVolume pvc-e031e017-c95c-4357-b2b0-ac63978d2248 to get deleted Jan 25 06:52:45.853: INFO: PersistentVolume pvc-e031e017-c95c-4357-b2b0-ac63978d2248 found and phase=Released (30.824757ms) Jan 25 06:52:50.884: INFO: PersistentVolume pvc-e031e017-c95c-4357-b2b0-ac63978d2248 found and phase=Released (5.062405234s) Jan 25 06:52:55.917: INFO: PersistentVolume pvc-e031e017-c95c-4357-b2b0-ac63978d2248 was removed �[1mSTEP:�[0m Deleting sc �[38;5;243m01/25/23 06:52:55.917�[0m [AfterEach] [Testpattern: Dynamic PV (block volmode)] volumeMode test/e2e/framework/node/init/init.go:32 Jan 25 06:52:55.950: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [Testpattern: Dynamic PV (block volmode)] volumeMode test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [Testpattern: Dynamic PV (block volmode)] volumeMode dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/25/23 06:52:55.982�[0m �[1mSTEP:�[0m Collecting events from namespace "volumemode-9184". �[38;5;243m01/25/23 06:52:55.982�[0m �[1mSTEP:�[0m Found 16 events. �[38;5;243m01/25/23 06:52:56.014�[0m Jan 25 06:52:56.014: INFO: At 2023-01-25 06:47:43 +0000 UTC - event for ebs.csi.aws.comrd5b5: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Jan 25 06:52:56.014: INFO: At 2023-01-25 06:47:43 +0000 UTC - event for ebs.csi.aws.comrd5b5: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Jan 25 06:52:56.014: INFO: At 2023-01-25 06:47:43 +0000 UTC - event for ebs.csi.aws.comrd5b5: {ebs.csi.aws.com_i-071cd767a01ed5a98_1fc23767-079d-4052-a310-50d34759c48e } Provisioning: External provisioner is provisioning volume for claim "volumemode-9184/ebs.csi.aws.comrd5b5" Jan 25 06:52:56.014: INFO: At 2023-01-25 06:47:46 +0000 UTC - event for ebs.csi.aws.comrd5b5: {ebs.csi.aws.com_i-071cd767a01ed5a98_1fc23767-079d-4052-a310-50d34759c48e } ProvisioningSucceeded: Successfully provisioned volume pvc-e031e017-c95c-4357-b2b0-ac63978d2248 Jan 25 06:52:56.014: INFO: At 2023-01-25 06:47:47 +0000 UTC - event for pod-c185a4fa-326c-4f33-957a-bddf24584c62: {default-scheduler } Scheduled: Successfully assigned volumemode-9184/pod-c185a4fa-326c-4f33-957a-bddf24584c62 to i-0e8434bc9e1236fd5 Jan 25 06:52:56.014: INFO: At 2023-01-25 06:47:47 +0000 UTC - event for pod-c185a4fa-326c-4f33-957a-bddf24584c62: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "dc1f4cb37cf0f770fa839cbac3c9eaaf2c4fa64076955653d5a9038eea74addf": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:52:56.014: INFO: At 2023-01-25 06:47:49 +0000 UTC - event for pod-c185a4fa-326c-4f33-957a-bddf24584c62: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-e031e017-c95c-4357-b2b0-ac63978d2248" Jan 25 06:52:56.014: INFO: At 2023-01-25 06:48:01 +0000 UTC - event for pod-c185a4fa-326c-4f33-957a-bddf24584c62: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d7fdf122a705e1b1f5082635acc4c7a0d77b1e772a0044366c9f9243121c4cb3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:52:56.014: INFO: At 2023-01-25 06:48:13 +0000 UTC - event for pod-c185a4fa-326c-4f33-957a-bddf24584c62: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6d361c8639134fac07ce0a7430a801e4a38877944a6908918f1995cfd3a12d7e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:52:56.014: INFO: At 2023-01-25 06:48:27 +0000 UTC - event for pod-c185a4fa-326c-4f33-957a-bddf24584c62: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "aa6758c9b01d2dcef58d1e0b393470dcfe20bee6d4ccdc68e37fc2441cda231a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:52:56.014: INFO: At 2023-01-25 06:48:39 +0000 UTC - event for pod-c185a4fa-326c-4f33-957a-bddf24584c62: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ce7532965d26df2e3966a41997eff0758b2d76c15c9afffb650c3d4100bd6960": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:52:56.014: INFO: At 2023-01-25 06:48:51 +0000 UTC - event for pod-c185a4fa-326c-4f33-957a-bddf24584c62: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a115c256adb0f4a0fda6c1cf7d697fbf6b9dba6a5a65a1e6e38febbc11d0db87": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:52:56.014: INFO: At 2023-01-25 06:49:05 +0000 UTC - event for pod-c185a4fa-326c-4f33-957a-bddf24584c62: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c45c3ea49cc18633e8d2e55678fb7955111d444038361cf1d2c67394de27b0c7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:52:56.014: INFO: At 2023-01-25 06:49:20 +0000 UTC - event for pod-c185a4fa-326c-4f33-957a-bddf24584c62: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5d87e45c26445ca56d408eebd5b48943db2ad8333f51a7830e60fbb906666729": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:52:56.014: INFO: At 2023-01-25 06:49:33 +0000 UTC - event for pod-c185a4fa-326c-4f33-957a-bddf24584c62: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c1cb314937d2a5110d73c49ed2cad3988eec1def4954a5ca4c3d9b2ee4cd9418": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:52:56.014: INFO: At 2023-01-25 06:49:46 +0000 UTC - event for pod-c185a4fa-326c-4f33-957a-bddf24584c62: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "254f17b9af18847b308316c6db7cbb73cd9680368e215bad797b76a5f68701c2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:52:56.047: INFO: POD NODE PHASE GRACE CONDITIONS Jan 25 06:52:56.047: INFO: Jan 25 06:52:56.079: INFO: Logging node info for node i-071cd767a01ed5a98 Jan 25 06:52:56.110: INFO: Node Info: &Node{ObjectMeta:{i-071cd767a01ed5a98 e176a57e-e197-4621-9ff9-1551f1909075 46278 0 2023-01-25 06:25:20 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-071cd767a01ed5a98 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ca-central-1a topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.39.108 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-071cd767a01ed5a98"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-25 06:25:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-25 06:25:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:26:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:26:12 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-25 06:26:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-25 06:48:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-071cd767a01ed5a98,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862937600 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758080000 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:48:09 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:48:09 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:48:09 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:48:09 +0000 UTC,LastTransitionTime:2023-01-25 06:26:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.108,},NodeAddress{Type:ExternalIP,Address:3.97.7.38,},NodeAddress{Type:InternalDNS,Address:i-071cd767a01ed5a98.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-071cd767a01ed5a98.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-97-7-38.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dc6c1622927717fff709d069d68b2,SystemUUID:ec2dc6c1-6229-2771-7fff-709d069d68b2,BootID:034bcfe8-9462-416d-9a80-bb9dcb9fd35d,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.1],SizeBytes:135178704,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.1],SizeBytes:124995897,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:63920476,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.1],SizeBytes:57661752,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:44339025,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42841295,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4965790,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 25 06:52:56.111: INFO: Logging kubelet events for node i-071cd767a01ed5a98 Jan 25 06:52:56.145: INFO: Logging pods the kubelet thinks is on node i-071cd767a01ed5a98 Jan 25 06:52:56.184: INFO: kube-controller-manager-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.184: INFO: Container kube-controller-manager ready: true, restart count 4 Jan 25 06:52:56.184: INFO: kube-scheduler-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.184: INFO: Container kube-scheduler ready: true, restart count 0 Jan 25 06:52:56.184: INFO: cilium-nqkjx started at 2023-01-25 06:26:00 +0000 UTC (1+1 container statuses recorded) Jan 25 06:52:56.184: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Container cilium-agent ready: true, restart count 0 Jan 25 06:52:56.184: INFO: dns-controller-69987775c6-7nx57 started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.184: INFO: Container dns-controller ready: true, restart count 0 Jan 25 06:52:56.184: INFO: aws-cloud-controller-manager-6kpfv started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.184: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 25 06:52:56.184: INFO: etcd-manager-events-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (10+1 container statuses recorded) Jan 25 06:52:56.184: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Container etcd-manager ready: true, restart count 0 Jan 25 06:52:56.184: INFO: etcd-manager-main-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (10+1 container statuses recorded) Jan 25 06:52:56.184: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Container etcd-manager ready: true, restart count 0 Jan 25 06:52:56.184: INFO: kube-apiserver-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+2 container statuses recorded) Jan 25 06:52:56.184: INFO: Container healthcheck ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Container kube-apiserver ready: true, restart count 3 Jan 25 06:52:56.184: INFO: cilium-operator-5dd44dc49f-jhvqd started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.184: INFO: Container cilium-operator ready: true, restart count 0 Jan 25 06:52:56.184: INFO: kops-controller-qfzp7 started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.184: INFO: Container kops-controller ready: true, restart count 0 Jan 25 06:52:56.184: INFO: ebs-csi-controller-5bd98b456f-cdjzf started at 2023-01-25 06:26:00 +0000 UTC (0+5 container statuses recorded) Jan 25 06:52:56.184: INFO: Container csi-attacher ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Container csi-provisioner ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Container csi-resizer ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Container ebs-plugin ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Container liveness-probe ready: true, restart count 0 Jan 25 06:52:56.184: INFO: ebs-csi-node-7t2hw started at 2023-01-25 06:26:00 +0000 UTC (0+3 container statuses recorded) Jan 25 06:52:56.184: INFO: Container ebs-plugin ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Container liveness-probe ready: true, restart count 0 Jan 25 06:52:56.184: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 25 06:52:56.339: INFO: Latency metrics for node i-071cd767a01ed5a98 Jan 25 06:52:56.339: INFO: Logging node info for node i-0dbcb4dc9e1f5e62b Jan 25 06:52:56.370: INFO: Node Info: &Node{ObjectMeta:{i-0dbcb4dc9e1f5e62b f1ad0cae-2ed9-41a4-8be5-9884ae32adaa 49056 0 2023-01-25 06:27:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0dbcb4dc9e1f5e62b kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0dbcb4dc9e1f5e62b topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.63.11 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0dbcb4dc9e1f5e62b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-25 06:46:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:52:16 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:52:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0dbcb4dc9e1f5e62b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:38 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.11,},NodeAddress{Type:ExternalIP,Address:3.96.138.144,},NodeAddress{Type:InternalDNS,Address:i-0dbcb4dc9e1f5e62b.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0dbcb4dc9e1f5e62b.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-138-144.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2cc8535b4596aa012ef7857efd2302,SystemUUID:ec2cc853-5b45-96aa-012e-f7857efd2302,BootID:1d85d5b3-ad1c-462b-98e2-f01921a4e990,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-08b8d4339aaccaee6],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-08b8d4339aaccaee6,DevicePath:,},},Config:nil,},} Jan 25 06:52:56.371: INFO: Logging kubelet events for node i-0dbcb4dc9e1f5e62b Jan 25 06:52:56.405: INFO: Logging pods the kubelet thinks is on node i-0dbcb4dc9e1f5e62b Jan 25 06:52:56.444: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.444: INFO: local-injector started at 2023-01-25 06:50:06 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container local-injector ready: false, restart count 0 Jan 25 06:52:56.444: INFO: pod-d973dfcb-fb6f-420f-9eb8-855583292025 started at 2023-01-25 06:51:32 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container test-container ready: false, restart count 0 Jan 25 06:52:56.444: INFO: cilium-wptjp started at 2023-01-25 06:27:26 +0000 UTC (1+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 25 06:52:56.444: INFO: Container cilium-agent ready: true, restart count 0 Jan 25 06:52:56.444: INFO: csi-mockplugin-attacher-0 started at 2023-01-25 06:50:16 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container csi-attacher ready: false, restart count 0 Jan 25 06:52:56.444: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jjlq8 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.444: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rt5t5 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.444: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-8h8sv started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.444: INFO: webserver-deployment-7f5969cbc7-fqq2n started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container httpd ready: true, restart count 0 Jan 25 06:52:56.444: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.444: INFO: csi-mockplugin-0 started at 2023-01-25 06:50:15 +0000 UTC (0+3 container statuses recorded) Jan 25 06:52:56.444: INFO: Container csi-provisioner ready: false, restart count 0 Jan 25 06:52:56.444: INFO: Container driver-registrar ready: false, restart count 0 Jan 25 06:52:56.444: INFO: Container mock ready: false, restart count 0 Jan 25 06:52:56.444: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.444: INFO: csi-mockplugin-resizer-0 started at 2023-01-25 06:50:16 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container csi-resizer ready: true, restart count 0 Jan 25 06:52:56.444: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-4wfp7 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.444: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-psvwb started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.444: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.444: INFO: hostexec-i-0dbcb4dc9e1f5e62b-97cc8 started at 2023-01-25 06:50:00 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container agnhost-container ready: true, restart count 0 Jan 25 06:52:56.444: INFO: webserver-deployment-7f5969cbc7-pzv6w started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container httpd ready: false, restart count 0 Jan 25 06:52:56.444: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xqz46 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.444: INFO: busybox-1b7825ea-2c0f-4fcb-b7f9-c97bd78ff608 started at 2023-01-25 06:50:33 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container busybox ready: false, restart count 0 Jan 25 06:52:56.444: INFO: webserver-deployment-7f5969cbc7-ctzws started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Container httpd ready: false, restart count 0 Jan 25 06:52:56.444: INFO: pod-subpath-test-dynamicpv-gwtk started at 2023-01-25 06:51:51 +0000 UTC (2+1 container statuses recorded) Jan 25 06:52:56.444: INFO: Init container init-volume-dynamicpv-gwtk ready: false, restart count 0 Jan 25 06:52:56.444: INFO: Init container test-init-volume-dynamicpv-gwtk ready: false, restart count 0 Jan 25 06:52:56.444: INFO: Container test-container-subpath-dynamicpv-gwtk ready: false, restart count 0 Jan 25 06:52:56.444: INFO: ebs-csi-node-bn5vq started at 2023-01-25 06:27:26 +0000 UTC (0+3 container statuses recorded) Jan 25 06:52:56.444: INFO: Container ebs-plugin ready: true, restart count 0 Jan 25 06:52:56.444: INFO: Container liveness-probe ready: true, restart count 0 Jan 25 06:52:56.444: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 25 06:52:56.647: INFO: Latency metrics for node i-0dbcb4dc9e1f5e62b Jan 25 06:52:56.647: INFO: Logging node info for node i-0df245ff74f46fa48 Jan 25 06:52:56.679: INFO: Node Info: &Node{ObjectMeta:{i-0df245ff74f46fa48 3893284f-3580-49f6-b7a1-2ddba9ae06ec 48407 0 2023-01-25 06:27:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0df245ff74f46fa48 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0df245ff74f46fa48 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.41.87 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-6879":"i-0df245ff74f46fa48","ebs.csi.aws.com":"i-0df245ff74f46fa48"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kops-controller Update v1 2023-01-25 06:27:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {e2e.test Update v1 2023-01-25 06:46:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:50:23 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:50:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0df245ff74f46fa48,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:40 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.87,},NodeAddress{Type:ExternalIP,Address:3.96.184.142,},NodeAddress{Type:InternalDNS,Address:i-0df245ff74f46fa48.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0df245ff74f46fa48.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-184-142.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2129fadd2b2b52f98b14d7c0337fff,SystemUUID:ec2129fa-dd2b-2b52-f98b-14d7c0337fff,BootID:6b9e799a-3b2a-44dc-8a4e-1a86f3b5230b,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-020105115fe204770],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-020105115fe204770,DevicePath:,},},Config:nil,},} Jan 25 06:52:56.680: INFO: Logging kubelet events for node i-0df245ff74f46fa48 Jan 25 06:52:56.714: INFO: Logging pods the kubelet thinks is on node i-0df245ff74f46fa48 Jan 25 06:52:56.755: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sxzm4 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.755: INFO: pvc-volume-tester-mrm72 started at 2023-01-25 06:52:23 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Container volume-tester ready: false, restart count 0 Jan 25 06:52:56.755: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.755: INFO: pfpod started at 2023-01-25 06:49:34 +0000 UTC (0+2 container statuses recorded) Jan 25 06:52:56.755: INFO: Container portforwardtester ready: false, restart count 0 Jan 25 06:52:56.755: INFO: Container readiness ready: false, restart count 0 Jan 25 06:52:56.755: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-569r5 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.755: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-plznd started at 2023-01-25 06:46:16 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.755: INFO: csi-mockplugin-0 started at 2023-01-25 06:46:38 +0000 UTC (0+3 container statuses recorded) Jan 25 06:52:56.755: INFO: Container csi-provisioner ready: true, restart count 0 Jan 25 06:52:56.755: INFO: Container driver-registrar ready: true, restart count 0 Jan 25 06:52:56.755: INFO: Container mock ready: true, restart count 0 Jan 25 06:52:56.755: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2mwzt started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.755: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.755: INFO: webserver-deployment-7f5969cbc7-q9lv4 started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Container httpd ready: false, restart count 0 Jan 25 06:52:56.755: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hd52p started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.755: INFO: webserver-deployment-7f5969cbc7-hrp6g started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Container httpd ready: false, restart count 0 Jan 25 06:52:56.755: INFO: cilium-4kl5t started at 2023-01-25 06:27:28 +0000 UTC (1+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 25 06:52:56.755: INFO: Container cilium-agent ready: true, restart count 0 Jan 25 06:52:56.755: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rzvq9 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.755: INFO: pod-b2d5aa59-ddb2-497b-b629-98cd25649a85 started at 2023-01-25 06:50:17 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Container write-pod ready: false, restart count 0 Jan 25 06:52:56.755: INFO: webserver-deployment-7f5969cbc7-r4gfp started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Container httpd ready: true, restart count 0 Jan 25 06:52:56.755: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-szf5f started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.755: INFO: ebs-csi-node-cfszw started at 2023-01-25 06:27:28 +0000 UTC (0+3 container statuses recorded) Jan 25 06:52:56.755: INFO: Container ebs-plugin ready: true, restart count 0 Jan 25 06:52:56.755: INFO: Container liveness-probe ready: true, restart count 0 Jan 25 06:52:56.755: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 25 06:52:56.755: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x5hz9 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:56.755: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:56.975: INFO: Latency metrics for node i-0df245ff74f46fa48 Jan 25 06:52:56.975: INFO: Logging node info for node i-0e8434bc9e1236fd5 Jan 25 06:52:57.006: INFO: Node Info: &Node{ObjectMeta:{i-0e8434bc9e1236fd5 48f670c2-fe1c-46d3-b197-464836dab89d 49218 0 2023-01-25 06:27:23 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0e8434bc9e1236fd5 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0e8434bc9e1236fd5 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.54.30 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-5972":"i-0e8434bc9e1236fd5","ebs.csi.aws.com":"i-0e8434bc9e1236fd5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-25 06:46:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-25 06:50:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-25 06:52:43 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0e8434bc9e1236fd5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:35 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.30,},NodeAddress{Type:ExternalIP,Address:99.79.58.56,},NodeAddress{Type:InternalDNS,Address:i-0e8434bc9e1236fd5.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0e8434bc9e1236fd5.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-99-79-58-56.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2439e74c65fcfb2f8c6ece88db3944,SystemUUID:ec2439e7-4c65-fcfb-2f8c-6ece88db3944,BootID:568b3843-a4d7-46ef-b502-19f3e8310af2,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-5972^782f81a1-9c7c-11ed-97c4-deff35c7067a kubernetes.io/csi/ebs.csi.aws.com^vol-058ab77a5848fba8a kubernetes.io/csi/ebs.csi.aws.com^vol-06d3e55226f87a3e7],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-06d3e55226f87a3e7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-058ab77a5848fba8a,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-5972^782f81a1-9c7c-11ed-97c4-deff35c7067a,DevicePath:,},},Config:nil,},} Jan 25 06:52:57.007: INFO: Logging kubelet events for node i-0e8434bc9e1236fd5 Jan 25 06:52:57.040: INFO: Logging pods the kubelet thinks is on node i-0e8434bc9e1236fd5 Jan 25 06:52:57.101: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-tczwz started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.101: INFO: csi-hostpathplugin-0 started at 2023-01-25 06:49:50 +0000 UTC (0+7 container statuses recorded) Jan 25 06:52:57.101: INFO: Container csi-attacher ready: true, restart count 0 Jan 25 06:52:57.101: INFO: Container csi-provisioner ready: true, restart count 0 Jan 25 06:52:57.101: INFO: Container csi-resizer ready: true, restart count 0 Jan 25 06:52:57.101: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 25 06:52:57.101: INFO: Container hostpath ready: true, restart count 0 Jan 25 06:52:57.101: INFO: Container liveness-probe ready: true, restart count 0 Jan 25 06:52:57.101: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 25 06:52:57.101: INFO: webserver-deployment-7f5969cbc7-t7ck4 started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container httpd ready: false, restart count 0 Jan 25 06:52:57.101: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sr5k2 started at 2023-01-25 06:46:16 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.101: INFO: csi-mockplugin-attacher-0 started at 2023-01-25 06:49:32 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container csi-attacher ready: false, restart count 0 Jan 25 06:52:57.101: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.101: INFO: pod-subpath-test-dynamicpv-mf6j started at 2023-01-25 06:49:53 +0000 UTC (0+2 container statuses recorded) Jan 25 06:52:57.101: INFO: Container test-container-subpath-dynamicpv-mf6j ready: false, restart count 0 Jan 25 06:52:57.101: INFO: Container test-container-volume-dynamicpv-mf6j ready: false, restart count 0 Jan 25 06:52:57.101: INFO: cilium-xkk5h started at 2023-01-25 06:27:24 +0000 UTC (1+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 25 06:52:57.101: INFO: Container cilium-agent ready: true, restart count 0 Jan 25 06:52:57.101: INFO: inline-volume-tester-2shhw started at 2023-01-25 06:47:57 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 25 06:52:57.101: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rbghp started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.101: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: false, restart count 0 Jan 25 06:52:57.101: INFO: pod-subpath-test-preprovisionedpv-vfbk started at 2023-01-25 06:51:06 +0000 UTC (1+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Init container init-volume-preprovisionedpv-vfbk ready: false, restart count 0 Jan 25 06:52:57.101: INFO: Container test-container-subpath-preprovisionedpv-vfbk ready: false, restart count 0 Jan 25 06:52:57.101: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2kf7l started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.101: INFO: coredns-559769c974-ggjhw started at 2023-01-25 06:27:35 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container coredns ready: true, restart count 0 Jan 25 06:52:57.101: INFO: hostexec-i-0e8434bc9e1236fd5-9ql96 started at 2023-01-25 06:50:49 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container agnhost-container ready: true, restart count 0 Jan 25 06:52:57.101: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5mlr8 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.101: INFO: hostexec-i-0e8434bc9e1236fd5-274ww started at 2023-01-25 06:50:47 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container agnhost-container ready: true, restart count 0 Jan 25 06:52:57.101: INFO: pod-9c25960d-410f-429b-b861-ab98bb2eb98b started at 2023-01-25 06:50:50 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container write-pod ready: false, restart count 0 Jan 25 06:52:57.101: INFO: webserver-deployment-7f5969cbc7-wwshs started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container httpd ready: false, restart count 0 Jan 25 06:52:57.101: INFO: ebs-csi-node-4d5tv started at 2023-01-25 06:27:24 +0000 UTC (0+3 container statuses recorded) Jan 25 06:52:57.101: INFO: Container ebs-plugin ready: true, restart count 0 Jan 25 06:52:57.101: INFO: Container liveness-probe ready: true, restart count 0 Jan 25 06:52:57.101: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 25 06:52:57.101: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qqns6 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.101: INFO: csi-mockplugin-0 started at 2023-01-25 06:49:32 +0000 UTC (0+3 container statuses recorded) Jan 25 06:52:57.101: INFO: Container csi-provisioner ready: false, restart count 0 Jan 25 06:52:57.101: INFO: Container driver-registrar ready: false, restart count 0 Jan 25 06:52:57.101: INFO: Container mock ready: false, restart count 0 Jan 25 06:52:57.101: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jspbp started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.101: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-lw7tl started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.101: INFO: coredns-autoscaler-7cb5c5b969-mndhj started at 2023-01-25 06:27:35 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.101: INFO: Container autoscaler ready: true, restart count 0 Jan 25 06:52:57.309: INFO: Latency metrics for node i-0e8434bc9e1236fd5 Jan 25 06:52:57.309: INFO: Logging node info for node i-0ebbe4bd7ec58d213 Jan 25 06:52:57.340: INFO: Node Info: &Node{ObjectMeta:{i-0ebbe4bd7ec58d213 f69d5727-b51d-4595-ab9a-fe57b5c9c863 47765 0 2023-01-25 06:27:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0ebbe4bd7ec58d213 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0ebbe4bd7ec58d213 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.215 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0ebbe4bd7ec58d213"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-25 06:46:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:49:53 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:49:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0ebbe4bd7ec58d213,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051685376 0} {<nil>} 3956724Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946827776 0} {<nil>} 3854324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:49:56 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:49:56 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:49:56 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:49:56 +0000 UTC,LastTransitionTime:2023-01-25 06:27:38 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.215,},NodeAddress{Type:ExternalIP,Address:3.96.126.35,},NodeAddress{Type:InternalDNS,Address:i-0ebbe4bd7ec58d213.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ebbe4bd7ec58d213.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-126-35.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2da1db9b923c887fdbe1dd4557a0d5,SystemUUID:ec2da1db-9b92-3c88-7fdb-e1dd4557a0d5,BootID:10184fa3-2e15-48fe-9718-382b9d68823e,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-03294c638eea79454 kubernetes.io/csi/ebs.csi.aws.com^vol-0d005d0140f2dadfb],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-03294c638eea79454,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d005d0140f2dadfb,DevicePath:,},},Config:nil,},} Jan 25 06:52:57.340: INFO: Logging kubelet events for node i-0ebbe4bd7ec58d213 Jan 25 06:52:57.374: INFO: Logging pods the kubelet thinks is on node i-0ebbe4bd7ec58d213 Jan 25 06:52:57.413: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hqhjx started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.413: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-csrp6 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.413: INFO: hostexec-i-0ebbe4bd7ec58d213-brbmd started at 2023-01-25 06:51:14 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container agnhost-container ready: true, restart count 0 Jan 25 06:52:57.413: INFO: ebs-csi-node-9ngsq started at 2023-01-25 06:27:27 +0000 UTC (0+3 container statuses recorded) Jan 25 06:52:57.413: INFO: Container ebs-plugin ready: true, restart count 0 Jan 25 06:52:57.413: INFO: Container liveness-probe ready: true, restart count 0 Jan 25 06:52:57.413: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 25 06:52:57.413: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bpq52 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.413: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qs6zz started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.413: INFO: pod-disruption-failure-ignore-0-l6qmf started at 2023-01-25 06:46:49 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container c ready: true, restart count 0 Jan 25 06:52:57.413: INFO: pod-disruption-failure-ignore-1-f77bm started at 2023-01-25 06:46:49 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container c ready: false, restart count 0 Jan 25 06:52:57.413: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-k9wft started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.413: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-npbz4 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.413: INFO: cilium-pkfh6 started at 2023-01-25 06:27:27 +0000 UTC (1+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 25 06:52:57.413: INFO: Container cilium-agent ready: true, restart count 0 Jan 25 06:52:57.413: INFO: coredns-559769c974-v87jf started at 2023-01-25 06:27:57 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container coredns ready: true, restart count 0 Jan 25 06:52:57.413: INFO: inline-volume-tester-bns44 started at 2023-01-25 06:49:50 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 25 06:52:57.413: INFO: webserver-deployment-7f5969cbc7-wch65 started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container httpd ready: false, restart count 0 Jan 25 06:52:57.413: INFO: pod-disruption-failure-ignore-1-vmrs8 started at 2023-01-25 06:47:22 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container c ready: false, restart count 0 Jan 25 06:52:57.413: INFO: csi-hostpathplugin-0 started at 2023-01-25 06:49:47 +0000 UTC (0+7 container statuses recorded) Jan 25 06:52:57.413: INFO: Container csi-attacher ready: false, restart count 0 Jan 25 06:52:57.413: INFO: Container csi-provisioner ready: false, restart count 0 Jan 25 06:52:57.413: INFO: Container csi-resizer ready: false, restart count 0 Jan 25 06:52:57.413: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 25 06:52:57.413: INFO: Container hostpath ready: false, restart count 0 Jan 25 06:52:57.413: INFO: Container liveness-probe ready: false, restart count 0 Jan 25 06:52:57.413: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 25 06:52:57.413: INFO: pod-7dc5023a-bc51-44e0-a5b1-2ac8cbb4d9a7 started at 2023-01-25 06:51:20 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container write-pod ready: false, restart count 0 Jan 25 06:52:57.413: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-wm7lb started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.413: INFO: pod-77e2c30c-da83-47c3-846b-f0c680a63e2d started at 2023-01-25 06:49:59 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container write-pod ready: false, restart count 0 Jan 25 06:52:57.413: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-7zg9l started at 2023-01-25 06:46:16 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.413: INFO: webserver-deployment-7f5969cbc7-wg5sj started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container httpd ready: false, restart count 0 Jan 25 06:52:57.413: INFO: external-injector started at 2023-01-25 06:49:33 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container external-injector ready: false, restart count 0 Jan 25 06:52:57.413: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x4pht started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.413: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ft4g2 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:52:57.413: INFO: hostexec-i-0ebbe4bd7ec58d213-9jht8 started at 2023-01-25 06:49:56 +0000 UTC (0+1 container statuses recorded) Jan 25 06:52:57.413: INFO: Container agnhost-container ready: true, restart count 0 Jan 25 06:52:57.676: INFO: Latency metrics for node i-0ebbe4bd7ec58d213 [DeferCleanup (Each)] [Testpattern: Dynamic PV (block volmode)] volumeMode tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "volumemode-9184" for this suite. �[38;5;243m01/25/23 06:52:57.676�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\svolumes\sshould\sstore\sdata$'
test/e2e/framework/volume/fixtures.go:571
k8s.io/kubernetes/test/e2e/framework/volume.InjectContent(0xc00180f860, {{0xc001840870, 0xa}, {0x75c79d6, 0x8}, {0x0, 0x0}, {0x0, 0x0, 0x0}, ...}, ...)
test/e2e/framework/volume/fixtures.go:571 +0x279
k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumesTestSuite).DefineTests.func3()
test/e2e/storage/testsuites/volumes.go:185 +0x457
[BeforeEach] [Testpattern: Dynamic PV (default fs)] volumes
test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (default fs)] volumes
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/25/23 06:49:29.395�[0m
Jan 25 06:49:29.395: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename volume �[38;5;243m01/25/23 06:49:29.396�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/25/23 06:49:29.49�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/25/23 06:49:29.551�[0m
[BeforeEach] [Testpattern: Dynamic PV (default fs)] volumes
test/e2e/framework/metrics/init/init.go:31
[It] should store data
test/e2e/storage/testsuites/volumes.go:158
Jan 25 06:49:29.612: INFO: Creating resource for dynamic PV
Jan 25 06:49:29.612: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi}
�[1mSTEP:�[0m creating a StorageClass volume-498-e2e-scl4lh9 �[38;5;243m01/25/23 06:49:29.612�[0m
�[1mSTEP:�[0m creating a claim �[38;5;243m01/25/23 06:49:29.644�[0m
Jan 25 06:49:29.644: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
�[1mSTEP:�[0m starting external-injector �[38;5;243m01/25/23 06:49:29.709�[0m
Jan 25 06:49:29.742: INFO: Waiting up to 5m0s for pod "external-injector" in namespace "volume-498" to be "running"
Jan 25 06:49:29.774: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 31.131233ms
Jan 25 06:49:31.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2.063241666s
Jan 25 06:49:33.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4.062376886s
Jan 25 06:49:35.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 6.062628358s
Jan 25 06:49:37.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 8.063323144s
Jan 25 06:49:39.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 10.062860454s
Jan 25 06:49:41.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 12.063184928s
Jan 25 06:49:43.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 14.063595999s
Jan 25 06:49:45.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 16.062993399s
Jan 25 06:49:47.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 18.063212298s
Jan 25 06:49:49.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 20.062704102s
Jan 25 06:49:51.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 22.062992327s
Jan 25 06:49:53.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 24.063979195s
Jan 25 06:49:55.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 26.063002649s
Jan 25 06:49:57.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 28.063407074s
Jan 25 06:49:59.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 30.06269319s
Jan 25 06:50:01.811: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 32.068117763s
Jan 25 06:50:03.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 34.062676343s
Jan 25 06:50:05.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 36.063188386s
Jan 25 06:50:07.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 38.063491574s
Jan 25 06:50:09.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 40.06347s
Jan 25 06:50:11.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 42.062726089s
Jan 25 06:50:13.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 44.062767218s
Jan 25 06:50:15.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 46.062681984s
Jan 25 06:50:17.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 48.063160894s
Jan 25 06:50:19.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 50.063288946s
Jan 25 06:50:21.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 52.063340523s
Jan 25 06:50:23.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 54.06267535s
Jan 25 06:50:25.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 56.062710876s
Jan 25 06:50:27.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 58.062869705s
Jan 25 06:50:29.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.063151593s
Jan 25 06:50:31.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.063978544s
Jan 25 06:50:33.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.062681118s
Jan 25 06:50:35.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.063097433s
Jan 25 06:50:37.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.063221135s
Jan 25 06:50:39.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.062997727s
Jan 25 06:50:41.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.063343579s
Jan 25 06:50:43.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.063636355s
Jan 25 06:50:45.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.063208014s
Jan 25 06:50:47.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.062551447s
Jan 25 06:50:49.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.062513124s
Jan 25 06:50:51.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.062949178s
Jan 25 06:50:53.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.063003119s
Jan 25 06:50:55.816: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.073629233s
Jan 25 06:50:57.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.063067687s
Jan 25 06:50:59.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.063057752s
Jan 25 06:51:01.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.063414405s
Jan 25 06:51:03.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.062935965s
Jan 25 06:51:05.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.063069973s
Jan 25 06:51:07.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.063231851s
Jan 25 06:51:09.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.062942495s
Jan 25 06:51:11.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.063311272s
Jan 25 06:51:13.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.063370045s
Jan 25 06:51:15.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.062353761s
Jan 25 06:51:17.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.063054214s
Jan 25 06:51:19.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.062947692s
Jan 25 06:51:21.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.06313117s
Jan 25 06:51:23.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.062279738s
Jan 25 06:51:25.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.062697514s
Jan 25 06:51:27.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.062788897s
Jan 25 06:51:29.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.062048794s
Jan 25 06:51:31.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.062987254s
Jan 25 06:51:33.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.062860951s
Jan 25 06:51:35.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.062983729s
Jan 25 06:51:37.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.062903408s
Jan 25 06:51:39.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.063479982s
Jan 25 06:51:41.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.063349925s
Jan 25 06:51:43.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.063326238s
Jan 25 06:51:45.809: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.066788811s
Jan 25 06:51:47.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.063081296s
Jan 25 06:51:49.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.063875795s
Jan 25 06:51:51.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.063918997s
Jan 25 06:51:53.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.062622367s
Jan 25 06:51:55.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.063158152s
Jan 25 06:51:57.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.063396206s
Jan 25 06:51:59.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.063154212s
Jan 25 06:52:01.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.063902621s
Jan 25 06:52:03.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.063050187s
Jan 25 06:52:05.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.063037557s
Jan 25 06:52:07.807: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.06405879s
Jan 25 06:52:09.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.06334375s
Jan 25 06:52:11.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.063089132s
Jan 25 06:52:13.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.062632853s
Jan 25 06:52:15.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.063022238s
Jan 25 06:52:17.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.063031365s
Jan 25 06:52:19.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.063171631s
Jan 25 06:52:21.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.062653574s
Jan 25 06:52:23.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.063386778s
Jan 25 06:52:25.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.06240865s
Jan 25 06:52:27.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.062859813s
Jan 25 06:52:29.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.063148355s
Jan 25 06:52:31.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.063290262s
Jan 25 06:52:33.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.063257417s
Jan 25 06:52:35.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.063199564s
Jan 25 06:52:37.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.063479795s
Jan 25 06:52:39.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.063013012s
Jan 25 06:52:41.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.062778821s
Jan 25 06:52:43.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.063666578s
Jan 25 06:52:45.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.062607027s
Jan 25 06:52:47.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.062677372s
Jan 25 06:52:49.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.062655595s
Jan 25 06:52:51.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.063156389s
Jan 25 06:52:53.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.062659005s
Jan 25 06:52:55.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.062864721s
Jan 25 06:52:57.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.062608587s
Jan 25 06:52:59.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.062751298s
Jan 25 06:53:01.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.063072814s
Jan 25 06:53:03.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.063827811s
Jan 25 06:53:05.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.062958212s
Jan 25 06:53:07.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.063263281s
Jan 25 06:53:09.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.063046887s
Jan 25 06:53:11.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.062878234s
Jan 25 06:53:13.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.062890373s
Jan 25 06:53:15.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.06294121s
Jan 25 06:53:17.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.062849476s
Jan 25 06:53:19.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.062718417s
Jan 25 06:53:21.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.063215184s
Jan 25 06:53:23.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.062522291s
Jan 25 06:53:25.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.062943989s
Jan 25 06:53:27.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.062587797s
Jan 25 06:53:29.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.062834193s
Jan 25 06:53:31.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.063133419s
Jan 25 06:53:33.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.062934775s
Jan 25 06:53:35.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.062864593s
Jan 25 06:53:37.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.0627578s
Jan 25 06:53:39.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.062880483s
Jan 25 06:53:41.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.062986259s
Jan 25 06:53:43.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.063261702s
Jan 25 06:53:45.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.063095674s
Jan 25 06:53:47.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.06314066s
Jan 25 06:53:49.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.062898704s
Jan 25 06:53:51.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.063399862s
Jan 25 06:53:53.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.063103809s
Jan 25 06:53:55.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.062920808s
Jan 25 06:53:57.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.063414421s
Jan 25 06:53:59.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.063404165s
Jan 25 06:54:01.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.063479173s
Jan 25 06:54:03.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.062901334s
Jan 25 06:54:05.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.06331131s
Jan 25 06:54:07.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.063950793s
Jan 25 06:54:09.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.063514369s
Jan 25 06:54:11.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.062888102s
Jan 25 06:54:13.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.063210551s
Jan 25 06:54:15.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.062986031s
Jan 25 06:54:17.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.063379364s
Jan 25 06:54:19.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.063260278s
Jan 25 06:54:21.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.062338434s
Jan 25 06:54:23.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.063286282s
Jan 25 06:54:25.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.062386155s
Jan 25 06:54:27.806: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.063984487s
Jan 25 06:54:29.805: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.062625398s
Jan 25 06:54:29.836: INFO: Pod "external-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.093684328s
�[1mSTEP:�[0m Deleting pod external-injector in namespace volume-498 �[38;5;243m01/25/23 06:54:29.836�[0m
Jan 25 06:54:29.871: INFO: Waiting for pod external-injector to disappear
Jan 25 06:54:29.902: INFO: Pod external-injector still exists
Jan 25 06:54:31.902: INFO: Waiting for pod external-injector to disappear
Jan 25 06:54:31.933: INFO: Pod external-injector no longer exists
Jan 25 06:54:31.933: FAIL: Failed to create injector pod: timed out while waiting for pod volume-498/external-injector to be running
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/volume.InjectContent(0xc00180f860, {{0xc001840870, 0xa}, {0x75c79d6, 0x8}, {0x0, 0x0}, {0x0, 0x0, 0x0}, ...}, ...)
test/e2e/framework/volume/fixtures.go:571 +0x279
k8s.io/kubernetes/test/e2e/storage/testsuites.(*volumesTestSuite).DefineTests.func3()
test/e2e/storage/testsuites/volumes.go:185 +0x457
�[1mSTEP:�[0m cleaning the environment after external �[38;5;243m01/25/23 06:54:31.933�[0m
�[1mSTEP:�[0m Deleting pvc �[38;5;243m01/25/23 06:54:31.933�[0m
Jan 25 06:54:31.996: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com6bbkl"
Jan 25 06:54:32.030: INFO: Waiting up to 5m0s for PersistentVolume pvc-f3141dcd-a9b3-42df-90cb-33d7c0e6110e to get deleted
Jan 25 06:54:32.061: INFO: PersistentVolume pvc-f3141dcd-a9b3-42df-90cb-33d7c0e6110e found and phase=Released (30.764646ms)
Jan 25 06:54:37.093: INFO: PersistentVolume pvc-f3141dcd-a9b3-42df-90cb-33d7c0e6110e found and phase=Released (5.062603251s)
Jan 25 06:54:42.124: INFO: PersistentVolume pvc-f3141dcd-a9b3-42df-90cb-33d7c0e6110e was removed
�[1mSTEP:�[0m Deleting sc �[38;5;243m01/25/23 06:54:42.124�[0m
[AfterEach] [Testpattern: Dynamic PV (default fs)] volumes
test/e2e/framework/node/init/init.go:32
Jan 25 06:54:42.158: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] volumes
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] volumes
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/25/23 06:54:42.19�[0m
�[1mSTEP:�[0m Collecting events from namespace "volume-498". �[38;5;243m01/25/23 06:54:42.19�[0m
�[1mSTEP:�[0m Found 16 events. �[38;5;243m01/25/23 06:54:42.222�[0m
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:49:29 +0000 UTC - event for ebs.csi.aws.com6bbkl: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:49:29 +0000 UTC - event for ebs.csi.aws.com6bbkl: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:49:29 +0000 UTC - event for ebs.csi.aws.com6bbkl: {ebs.csi.aws.com_i-071cd767a01ed5a98_1fc23767-079d-4052-a310-50d34759c48e } Provisioning: External provisioner is provisioning volume for claim "volume-498/ebs.csi.aws.com6bbkl"
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:49:33 +0000 UTC - event for ebs.csi.aws.com6bbkl: {ebs.csi.aws.com_i-071cd767a01ed5a98_1fc23767-079d-4052-a310-50d34759c48e } ProvisioningSucceeded: Successfully provisioned volume pvc-f3141dcd-a9b3-42df-90cb-33d7c0e6110e
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:49:33 +0000 UTC - event for external-injector: {default-scheduler } Scheduled: Successfully assigned volume-498/external-injector to i-0ebbe4bd7ec58d213
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:49:36 +0000 UTC - event for external-injector: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-f3141dcd-a9b3-42df-90cb-33d7c0e6110e"
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:49:36 +0000 UTC - event for external-injector: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cf9b2bb639ef091f2d2e3fe73915da98bd3840641f59e2a5bec7a6de03e9b67a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:49:52 +0000 UTC - event for external-injector: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "07547149862129685c8d0668b13846e4cbb7f7e3490b69290b5f81106ce7a517": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:50:06 +0000 UTC - event for external-injector: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "92d20e309d037b336306b2850abc33150dffbe49917fd3ab5cd653d782555735": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:50:21 +0000 UTC - event for external-injector: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3cc821200900f55893090dcea4d2a54e70754e6e06107bd1b7514a963d474a4a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:50:35 +0000 UTC - event for external-injector: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b0eee2896414a9ff33b9b5a96ecc321bfcc0d4fae08ec4c176c65d95de4a021d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:50:47 +0000 UTC - event for external-injector: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f99909335c8dd8a309d20e8167ad46efff4c2e70b3a3da1693acba5e012bb3f8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:51:02 +0000 UTC - event for external-injector: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b05dfd72c6439817f3b6af98bea2203bab191dde435e15ff36f30b1cf7eacf11": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:51:13 +0000 UTC - event for external-injector: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cb246f5c5ada3cb7dbf5151d5850e140eb7323de801b2e14dd1c1b6179a1bbf2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:51:26 +0000 UTC - event for external-injector: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b6e8c72ed138ded77dce8b447b1119e88ce5d7725c8a5c3ae728f12e41fe1b6f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:42.222: INFO: At 2023-01-25 06:51:42 +0000 UTC - event for external-injector: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0e700b67bbecc2008e79a7ae61574e1f81f9a64be08eef87ec32e56fdf9aabdf": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:42.252: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 25 06:54:42.252: INFO:
Jan 25 06:54:42.285: INFO:
Logging node info for node i-071cd767a01ed5a98
Jan 25 06:54:42.316: INFO: Node Info: &Node{ObjectMeta:{i-071cd767a01ed5a98 e176a57e-e197-4621-9ff9-1551f1909075 49349 0 2023-01-25 06:25:20 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-071cd767a01ed5a98 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ca-central-1a topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.39.108 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-071cd767a01ed5a98"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-25 06:25:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-25 06:25:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:26:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:26:12 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-25 06:26:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-25 06:53:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-071cd767a01ed5a98,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862937600 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758080000 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:26:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.108,},NodeAddress{Type:ExternalIP,Address:3.97.7.38,},NodeAddress{Type:InternalDNS,Address:i-071cd767a01ed5a98.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-071cd767a01ed5a98.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-97-7-38.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dc6c1622927717fff709d069d68b2,SystemUUID:ec2dc6c1-6229-2771-7fff-709d069d68b2,BootID:034bcfe8-9462-416d-9a80-bb9dcb9fd35d,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.1],SizeBytes:135178704,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.1],SizeBytes:124995897,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:63920476,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.1],SizeBytes:57661752,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:44339025,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42841295,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4965790,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 25 06:54:42.316: INFO:
Logging kubelet events for node i-071cd767a01ed5a98
Jan 25 06:54:42.351: INFO:
Logging pods the kubelet thinks is on node i-071cd767a01ed5a98
Jan 25 06:54:42.400: INFO: cilium-nqkjx started at 2023-01-25 06:26:00 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:54:42.400: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:54:42.400: INFO: dns-controller-69987775c6-7nx57 started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:42.400: INFO: Container dns-controller ready: true, restart count 0
Jan 25 06:54:42.400: INFO: aws-cloud-controller-manager-6kpfv started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:42.400: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 25 06:54:42.400: INFO: etcd-manager-events-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (10+1 container statuses recorded)
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Container etcd-manager ready: true, restart count 0
Jan 25 06:54:42.400: INFO: etcd-manager-main-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (10+1 container statuses recorded)
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Container etcd-manager ready: true, restart count 0
Jan 25 06:54:42.400: INFO: kube-apiserver-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+2 container statuses recorded)
Jan 25 06:54:42.400: INFO: Container healthcheck ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Container kube-apiserver ready: true, restart count 3
Jan 25 06:54:42.400: INFO: kube-controller-manager-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:42.400: INFO: Container kube-controller-manager ready: true, restart count 4
Jan 25 06:54:42.400: INFO: kube-scheduler-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:42.400: INFO: Container kube-scheduler ready: true, restart count 0
Jan 25 06:54:42.400: INFO: kops-controller-qfzp7 started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:42.400: INFO: Container kops-controller ready: true, restart count 0
Jan 25 06:54:42.400: INFO: ebs-csi-controller-5bd98b456f-cdjzf started at 2023-01-25 06:26:00 +0000 UTC (0+5 container statuses recorded)
Jan 25 06:54:42.400: INFO: Container csi-attacher ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Container csi-resizer ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:54:42.400: INFO: ebs-csi-node-7t2hw started at 2023-01-25 06:26:00 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:42.400: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:54:42.400: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:54:42.400: INFO: cilium-operator-5dd44dc49f-jhvqd started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:42.400: INFO: Container cilium-operator ready: true, restart count 0
Jan 25 06:54:42.563: INFO:
Latency metrics for node i-071cd767a01ed5a98
Jan 25 06:54:42.563: INFO:
Logging node info for node i-0dbcb4dc9e1f5e62b
Jan 25 06:54:42.594: INFO: Node Info: &Node{ObjectMeta:{i-0dbcb4dc9e1f5e62b f1ad0cae-2ed9-41a4-8be5-9884ae32adaa 49600 0 2023-01-25 06:27:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0dbcb4dc9e1f5e62b kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0dbcb4dc9e1f5e62b topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.63.11 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0dbcb4dc9e1f5e62b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:52:16 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:52:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0dbcb4dc9e1f5e62b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:38 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.11,},NodeAddress{Type:ExternalIP,Address:3.96.138.144,},NodeAddress{Type:InternalDNS,Address:i-0dbcb4dc9e1f5e62b.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0dbcb4dc9e1f5e62b.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-138-144.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2cc8535b4596aa012ef7857efd2302,SystemUUID:ec2cc853-5b45-96aa-012e-f7857efd2302,BootID:1d85d5b3-ad1c-462b-98e2-f01921a4e990,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-08b8d4339aaccaee6],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-08b8d4339aaccaee6,DevicePath:,},},Config:nil,},}
Jan 25 06:54:42.595: INFO:
Logging kubelet events for node i-0dbcb4dc9e1f5e62b
Jan 25 06:54:42.629: INFO:
Logging pods the kubelet thinks is on node i-0dbcb4dc9e1f5e62b
Jan 25 06:54:42.665: INFO: csi-mockplugin-resizer-0 started at 2023-01-25 06:50:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:42.665: INFO: Container csi-resizer ready: true, restart count 0
Jan 25 06:54:42.665: INFO: webserver-deployment-7f5969cbc7-pzv6w started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:42.665: INFO: Container httpd ready: false, restart count 0
Jan 25 06:54:42.665: INFO: hostexec-i-0dbcb4dc9e1f5e62b-97cc8 started at 2023-01-25 06:50:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:42.665: INFO: Container agnhost-container ready: true, restart count 0
Jan 25 06:54:42.665: INFO: pod-subpath-test-dynamicpv-gwtk started at 2023-01-25 06:51:51 +0000 UTC (2+1 container statuses recorded)
Jan 25 06:54:42.665: INFO: Init container init-volume-dynamicpv-gwtk ready: false, restart count 0
Jan 25 06:54:42.665: INFO: Init container test-init-volume-dynamicpv-gwtk ready: false, restart count 0
Jan 25 06:54:42.665: INFO: Container test-container-subpath-dynamicpv-gwtk ready: false, restart count 0
Jan 25 06:54:42.665: INFO: ebs-csi-node-bn5vq started at 2023-01-25 06:27:26 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:42.665: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:54:42.665: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:54:42.665: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:54:42.665: INFO: busybox-1b7825ea-2c0f-4fcb-b7f9-c97bd78ff608 started at 2023-01-25 06:50:33 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:42.665: INFO: Container busybox ready: true, restart count 0
Jan 25 06:54:42.665: INFO: webserver-deployment-7f5969cbc7-ctzws started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:42.665: INFO: Container httpd ready: true, restart count 0
Jan 25 06:54:42.665: INFO: cilium-wptjp started at 2023-01-25 06:27:26 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:54:42.665: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:54:42.665: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:54:42.665: INFO: local-injector started at 2023-01-25 06:50:06 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:42.665: INFO: Container local-injector ready: true, restart count 0
Jan 25 06:54:42.665: INFO: pod-d973dfcb-fb6f-420f-9eb8-855583292025 started at 2023-01-25 06:51:32 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:42.665: INFO: Container test-container ready: false, restart count 0
Jan 25 06:54:42.665: INFO: webserver-deployment-7f5969cbc7-fqq2n started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:42.665: INFO: Container httpd ready: true, restart count 0
Jan 25 06:54:42.665: INFO: csi-mockplugin-attacher-0 started at 2023-01-25 06:50:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:42.665: INFO: Container csi-attacher ready: false, restart count 0
Jan 25 06:54:42.665: INFO: csi-mockplugin-0 started at 2023-01-25 06:50:15 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:42.665: INFO: Container csi-provisioner ready: false, restart count 0
Jan 25 06:54:42.665: INFO: Container driver-registrar ready: false, restart count 0
Jan 25 06:54:42.665: INFO: Container mock ready: false, restart count 0
Jan 25 06:54:42.916: INFO:
Latency metrics for node i-0dbcb4dc9e1f5e62b
Jan 25 06:54:42.916: INFO:
Logging node info for node i-0df245ff74f46fa48
Jan 25 06:54:42.947: INFO: Node Info: &Node{ObjectMeta:{i-0df245ff74f46fa48 3893284f-3580-49f6-b7a1-2ddba9ae06ec 49602 0 2023-01-25 06:27:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0df245ff74f46fa48 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0df245ff74f46fa48 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.41.87 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-6879":"i-0df245ff74f46fa48","ebs.csi.aws.com":"i-0df245ff74f46fa48"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kops-controller Update v1 2023-01-25 06:27:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:50:23 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:50:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0df245ff74f46fa48,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:40 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.87,},NodeAddress{Type:ExternalIP,Address:3.96.184.142,},NodeAddress{Type:InternalDNS,Address:i-0df245ff74f46fa48.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0df245ff74f46fa48.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-184-142.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2129fadd2b2b52f98b14d7c0337fff,SystemUUID:ec2129fa-dd2b-2b52-f98b-14d7c0337fff,BootID:6b9e799a-3b2a-44dc-8a4e-1a86f3b5230b,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-020105115fe204770],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-020105115fe204770,DevicePath:,},},Config:nil,},}
Jan 25 06:54:42.948: INFO:
Logging kubelet events for node i-0df245ff74f46fa48
Jan 25 06:54:42.982: INFO:
Logging pods the kubelet thinks is on node i-0df245ff74f46fa48
Jan 25 06:54:43.018: INFO: csi-mockplugin-0 started at 2023-01-25 06:46:38 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:43.018: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:54:43.018: INFO: Container driver-registrar ready: true, restart count 0
Jan 25 06:54:43.018: INFO: Container mock ready: true, restart count 0
Jan 25 06:54:43.018: INFO: webserver-deployment-7f5969cbc7-q9lv4 started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.018: INFO: Container httpd ready: false, restart count 0
Jan 25 06:54:43.018: INFO: webserver-deployment-7f5969cbc7-hrp6g started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.018: INFO: Container httpd ready: false, restart count 0
Jan 25 06:54:43.018: INFO: cilium-4kl5t started at 2023-01-25 06:27:28 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:54:43.018: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:54:43.018: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:54:43.018: INFO: pod-b2d5aa59-ddb2-497b-b629-98cd25649a85 started at 2023-01-25 06:50:17 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.018: INFO: Container write-pod ready: false, restart count 0
Jan 25 06:54:43.018: INFO: webserver-deployment-7f5969cbc7-r4gfp started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.018: INFO: Container httpd ready: true, restart count 0
Jan 25 06:54:43.018: INFO: ebs-csi-node-cfszw started at 2023-01-25 06:27:28 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:43.018: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:54:43.018: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:54:43.018: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:54:43.018: INFO: pfpod started at 2023-01-25 06:49:34 +0000 UTC (0+2 container statuses recorded)
Jan 25 06:54:43.018: INFO: Container portforwardtester ready: true, restart count 0
Jan 25 06:54:43.018: INFO: Container readiness ready: false, restart count 0
Jan 25 06:54:43.241: INFO:
Latency metrics for node i-0df245ff74f46fa48
Jan 25 06:54:43.241: INFO:
Logging node info for node i-0e8434bc9e1236fd5
Jan 25 06:54:43.273: INFO: Node Info: &Node{ObjectMeta:{i-0e8434bc9e1236fd5 48f670c2-fe1c-46d3-b197-464836dab89d 49877 0 2023-01-25 06:27:23 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0e8434bc9e1236fd5 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0e8434bc9e1236fd5 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.54.30 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-5972":"i-0e8434bc9e1236fd5","csi-mock-csi-mock-volumes-7584":"i-0e8434bc9e1236fd5","ebs.csi.aws.com":"i-0e8434bc9e1236fd5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:52:43 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:54:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0e8434bc9e1236fd5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:35 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.30,},NodeAddress{Type:ExternalIP,Address:99.79.58.56,},NodeAddress{Type:InternalDNS,Address:i-0e8434bc9e1236fd5.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0e8434bc9e1236fd5.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-99-79-58-56.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2439e74c65fcfb2f8c6ece88db3944,SystemUUID:ec2439e7-4c65-fcfb-2f8c-6ece88db3944,BootID:568b3843-a4d7-46ef-b502-19f3e8310af2,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-5972^782f81a1-9c7c-11ed-97c4-deff35c7067a kubernetes.io/csi/ebs.csi.aws.com^vol-058ab77a5848fba8a kubernetes.io/csi/ebs.csi.aws.com^vol-06d3e55226f87a3e7],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-06d3e55226f87a3e7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-058ab77a5848fba8a,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-5972^782f81a1-9c7c-11ed-97c4-deff35c7067a,DevicePath:,},},Config:nil,},}
Jan 25 06:54:43.273: INFO:
Logging kubelet events for node i-0e8434bc9e1236fd5
Jan 25 06:54:43.307: INFO:
Logging pods the kubelet thinks is on node i-0e8434bc9e1236fd5
Jan 25 06:54:43.346: INFO: ebs-csi-node-4d5tv started at 2023-01-25 06:27:24 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:54:43.346: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:54:43.346: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:54:43.346: INFO: csi-mockplugin-0 started at 2023-01-25 06:49:32 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container csi-provisioner ready: false, restart count 0
Jan 25 06:54:43.346: INFO: Container driver-registrar ready: false, restart count 0
Jan 25 06:54:43.346: INFO: Container mock ready: false, restart count 0
Jan 25 06:54:43.346: INFO: hostexec-i-0e8434bc9e1236fd5-274ww started at 2023-01-25 06:50:47 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container agnhost-container ready: true, restart count 0
Jan 25 06:54:43.346: INFO: pod-9c25960d-410f-429b-b861-ab98bb2eb98b started at 2023-01-25 06:50:50 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container write-pod ready: false, restart count 0
Jan 25 06:54:43.346: INFO: webserver-deployment-7f5969cbc7-wwshs started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container httpd ready: false, restart count 0
Jan 25 06:54:43.346: INFO: coredns-autoscaler-7cb5c5b969-mndhj started at 2023-01-25 06:27:35 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container autoscaler ready: true, restart count 0
Jan 25 06:54:43.346: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-lw7tl started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:43.346: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sr5k2 started at 2023-01-25 06:46:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:43.346: INFO: csi-mockplugin-attacher-0 started at 2023-01-25 06:49:32 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container csi-attacher ready: false, restart count 0
Jan 25 06:54:43.346: INFO: csi-hostpathplugin-0 started at 2023-01-25 06:49:50 +0000 UTC (0+7 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container csi-attacher ready: true, restart count 0
Jan 25 06:54:43.346: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:54:43.346: INFO: Container csi-resizer ready: true, restart count 0
Jan 25 06:54:43.346: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 25 06:54:43.346: INFO: Container hostpath ready: true, restart count 0
Jan 25 06:54:43.346: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:54:43.346: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:54:43.346: INFO: webserver-deployment-7f5969cbc7-t7ck4 started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container httpd ready: false, restart count 0
Jan 25 06:54:43.346: INFO: cilium-xkk5h started at 2023-01-25 06:27:24 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:54:43.346: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:54:43.346: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:54:43.346: INFO: inline-volume-tester-2shhw started at 2023-01-25 06:47:57 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container csi-volume-tester ready: true, restart count 0
Jan 25 06:54:43.346: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rbghp started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:43.346: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:43.346: INFO: pod-subpath-test-dynamicpv-mf6j started at 2023-01-25 06:49:53 +0000 UTC (0+2 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container test-container-subpath-dynamicpv-mf6j ready: false, restart count 0
Jan 25 06:54:43.346: INFO: Container test-container-volume-dynamicpv-mf6j ready: false, restart count 0
Jan 25 06:54:43.346: INFO: pod-subpath-test-preprovisionedpv-vfbk started at 2023-01-25 06:51:06 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:54:43.346: INFO: Init container init-volume-preprovisionedpv-vfbk ready: true, restart count 0
Jan 25 06:54:43.346: INFO: Container test-container-subpath-preprovisionedpv-vfbk ready: false, restart count 0
Jan 25 06:54:43.346: INFO: coredns-559769c974-ggjhw started at 2023-01-25 06:27:35 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container coredns ready: true, restart count 0
Jan 25 06:54:43.346: INFO: hostexec-i-0e8434bc9e1236fd5-9ql96 started at 2023-01-25 06:50:49 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.346: INFO: Container agnhost-container ready: true, restart count 0
Jan 25 06:54:43.552: INFO:
Latency metrics for node i-0e8434bc9e1236fd5
Jan 25 06:54:43.552: INFO:
Logging node info for node i-0ebbe4bd7ec58d213
Jan 25 06:54:43.583: INFO: Node Info: &Node{ObjectMeta:{i-0ebbe4bd7ec58d213 f69d5727-b51d-4595-ab9a-fe57b5c9c863 49639 0 2023-01-25 06:27:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0ebbe4bd7ec58d213 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0ebbe4bd7ec58d213 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.215 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0ebbe4bd7ec58d213"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:54:31 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:54:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0ebbe4bd7ec58d213,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051685376 0} {<nil>} 3956724Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946827776 0} {<nil>} 3854324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:54:31 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:54:31 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:54:31 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:54:31 +0000 UTC,LastTransitionTime:2023-01-25 06:27:38 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.215,},NodeAddress{Type:ExternalIP,Address:3.96.126.35,},NodeAddress{Type:InternalDNS,Address:i-0ebbe4bd7ec58d213.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ebbe4bd7ec58d213.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-126-35.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2da1db9b923c887fdbe1dd4557a0d5,SystemUUID:ec2da1db-9b92-3c88-7fdb-e1dd4557a0d5,BootID:10184fa3-2e15-48fe-9718-382b9d68823e,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d005d0140f2dadfb],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d005d0140f2dadfb,DevicePath:,},},Config:nil,},}
Jan 25 06:54:43.583: INFO:
Logging kubelet events for node i-0ebbe4bd7ec58d213
Jan 25 06:54:43.618: INFO:
Logging pods the kubelet thinks is on node i-0ebbe4bd7ec58d213
Jan 25 06:54:43.658: INFO: pod-77e2c30c-da83-47c3-846b-f0c680a63e2d started at 2023-01-25 06:49:59 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.658: INFO: Container write-pod ready: false, restart count 0
Jan 25 06:54:43.658: INFO: webserver-deployment-7f5969cbc7-wg5sj started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.658: INFO: Container httpd ready: false, restart count 0
Jan 25 06:54:43.658: INFO: hostexec-i-0ebbe4bd7ec58d213-9jht8 started at 2023-01-25 06:49:56 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.658: INFO: Container agnhost-container ready: true, restart count 0
Jan 25 06:54:43.658: INFO: hostexec-i-0ebbe4bd7ec58d213-brbmd started at 2023-01-25 06:51:14 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.658: INFO: Container agnhost-container ready: true, restart count 0
Jan 25 06:54:43.658: INFO: ebs-csi-node-9ngsq started at 2023-01-25 06:27:27 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:43.658: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:54:43.658: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:54:43.658: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:54:43.658: INFO: pod-disruption-failure-ignore-0-l6qmf started at 2023-01-25 06:46:49 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.658: INFO: Container c ready: true, restart count 0
Jan 25 06:54:43.658: INFO: pod-disruption-failure-ignore-1-f77bm started at 2023-01-25 06:46:49 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.658: INFO: Container c ready: false, restart count 0
Jan 25 06:54:43.658: INFO: cilium-pkfh6 started at 2023-01-25 06:27:27 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:54:43.658: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:54:43.658: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:54:43.658: INFO: coredns-559769c974-v87jf started at 2023-01-25 06:27:57 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.658: INFO: Container coredns ready: true, restart count 0
Jan 25 06:54:43.658: INFO: inline-volume-tester-bns44 started at 2023-01-25 06:49:50 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.658: INFO: Container csi-volume-tester ready: true, restart count 0
Jan 25 06:54:43.658: INFO: webserver-deployment-7f5969cbc7-wch65 started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.658: INFO: Container httpd ready: true, restart count 0
Jan 25 06:54:43.658: INFO: pod-disruption-failure-ignore-1-vmrs8 started at 2023-01-25 06:47:22 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.658: INFO: Container c ready: false, restart count 0
Jan 25 06:54:43.658: INFO: csi-hostpathplugin-0 started at 2023-01-25 06:49:47 +0000 UTC (0+7 container statuses recorded)
Jan 25 06:54:43.658: INFO: Container csi-attacher ready: false, restart count 0
Jan 25 06:54:43.658: INFO: Container csi-provisioner ready: false, restart count 0
Jan 25 06:54:43.658: INFO: Container csi-resizer ready: false, restart count 0
Jan 25 06:54:43.658: INFO: Container csi-snapshotter ready: false, restart count 0
Jan 25 06:54:43.658: INFO: Container hostpath ready: false, restart count 0
Jan 25 06:54:43.658: INFO: Container liveness-probe ready: false, restart count 0
Jan 25 06:54:43.658: INFO: Container node-driver-registrar ready: false, restart count 0
Jan 25 06:54:43.658: INFO: pod-7dc5023a-bc51-44e0-a5b1-2ac8cbb4d9a7 started at 2023-01-25 06:51:20 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.658: INFO: Container write-pod ready: false, restart count 0
Jan 25 06:54:43.658: INFO: pod-disruption-failure-ignore-2-fv6kt started at 2023-01-25 06:54:42 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:43.658: INFO: Container c ready: false, restart count 0
Jan 25 06:54:43.945: INFO:
Latency metrics for node i-0ebbe4bd7ec58d213
[DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] volumes
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "volume-498" for this suite. �[38;5;243m01/25/23 06:54:43.945�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sPort\sforwarding\sWith\sa\sserver\slistening\son\s0\.0\.0\.0\sshould\ssupport\sforwarding\sover\swebsockets$'
test/e2e/kubectl/portforward.go:372
k8s.io/kubernetes/test/e2e/kubectl.doTestOverWebSockets({0x75c24b2, 0x7}, 0xc000c983c0)
test/e2e/kubectl/portforward.go:372 +0x2af
k8s.io/kubernetes/test/e2e/kubectl.glob..func2.1.3()
test/e2e/kubectl/portforward.go:471 +0x29
[BeforeEach] [sig-cli] Kubectl Port forwarding
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/25/23 06:49:34.041�[0m
Jan 25 06:49:34.041: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename port-forwarding �[38;5;243m01/25/23 06:49:34.042�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/25/23 06:49:34.136�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/25/23 06:49:34.197�[0m
[BeforeEach] [sig-cli] Kubectl Port forwarding
test/e2e/framework/metrics/init/init.go:31
[It] should support forwarding over websockets
test/e2e/kubectl/portforward.go:470
Jan 25 06:49:34.257: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Creating the pod �[38;5;243m01/25/23 06:49:34.258�[0m
Jan 25 06:49:34.294: INFO: Waiting up to 5m0s for pod "pfpod" in namespace "port-forwarding-5701" to be "running and ready"
Jan 25 06:49:34.324: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 30.798212ms
Jan 25 06:49:34.324: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:49:36.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2.0621569s
Jan 25 06:49:36.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:49:38.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4.062305164s
Jan 25 06:49:38.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:49:40.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 6.06312614s
Jan 25 06:49:40.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:49:42.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 8.062228364s
Jan 25 06:49:42.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:49:44.355: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 10.061870476s
Jan 25 06:49:44.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:49:46.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 12.062329465s
Jan 25 06:49:46.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:49:48.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 14.062972709s
Jan 25 06:49:48.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:49:50.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 16.063658688s
Jan 25 06:49:50.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:49:52.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 18.063746874s
Jan 25 06:49:52.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:49:54.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 20.063325478s
Jan 25 06:49:54.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:49:56.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 22.062147508s
Jan 25 06:49:56.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:49:58.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 24.062522557s
Jan 25 06:49:58.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:00.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 26.063032946s
Jan 25 06:50:00.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:02.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 28.062671722s
Jan 25 06:50:02.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:04.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 30.062516317s
Jan 25 06:50:04.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:06.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 32.063500213s
Jan 25 06:50:06.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:08.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 34.063258821s
Jan 25 06:50:08.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:10.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 36.062321563s
Jan 25 06:50:10.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:12.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 38.06275516s
Jan 25 06:50:12.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:14.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 40.062617662s
Jan 25 06:50:14.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:16.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 42.062099646s
Jan 25 06:50:16.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:18.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 44.062137314s
Jan 25 06:50:18.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:20.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 46.063048167s
Jan 25 06:50:20.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:22.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 48.06279916s
Jan 25 06:50:22.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:24.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 50.062946292s
Jan 25 06:50:24.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:26.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 52.06275825s
Jan 25 06:50:26.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:28.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 54.062161472s
Jan 25 06:50:28.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:30.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 56.063158382s
Jan 25 06:50:30.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:32.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 58.062587416s
Jan 25 06:50:32.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:34.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.062783184s
Jan 25 06:50:34.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:36.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.062744138s
Jan 25 06:50:36.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:38.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.062264772s
Jan 25 06:50:38.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:40.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.06337962s
Jan 25 06:50:40.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:42.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.063589128s
Jan 25 06:50:42.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:44.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.062021482s
Jan 25 06:50:44.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:46.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.062186279s
Jan 25 06:50:46.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:48.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.063572819s
Jan 25 06:50:48.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:50.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.06260885s
Jan 25 06:50:50.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:52.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.061944371s
Jan 25 06:50:52.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:54.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.062456391s
Jan 25 06:50:54.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:56.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.06355201s
Jan 25 06:50:56.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:50:58.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.063042185s
Jan 25 06:50:58.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:00.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.063131636s
Jan 25 06:51:00.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:02.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.062679689s
Jan 25 06:51:02.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:04.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.063069126s
Jan 25 06:51:04.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:06.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.062501326s
Jan 25 06:51:06.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:08.359: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.06575958s
Jan 25 06:51:08.359: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:10.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.062517206s
Jan 25 06:51:10.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:12.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.062996907s
Jan 25 06:51:12.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:14.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.062684774s
Jan 25 06:51:14.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:16.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.062256627s
Jan 25 06:51:16.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:18.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.062955026s
Jan 25 06:51:18.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:20.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.062676793s
Jan 25 06:51:20.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:22.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.062438809s
Jan 25 06:51:22.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:24.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.06265501s
Jan 25 06:51:24.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:26.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.062488475s
Jan 25 06:51:26.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:28.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.061995694s
Jan 25 06:51:28.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:30.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.063704381s
Jan 25 06:51:30.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:32.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.06242697s
Jan 25 06:51:32.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:34.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.063699499s
Jan 25 06:51:34.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:36.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.063461388s
Jan 25 06:51:36.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:38.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.062918678s
Jan 25 06:51:38.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:40.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.062517377s
Jan 25 06:51:40.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:42.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.063330066s
Jan 25 06:51:42.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:44.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.062452663s
Jan 25 06:51:44.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:46.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.06228829s
Jan 25 06:51:46.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:48.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.062901081s
Jan 25 06:51:48.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:50.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.063567489s
Jan 25 06:51:50.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:52.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.063779445s
Jan 25 06:51:52.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:54.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.063215693s
Jan 25 06:51:54.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:56.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.063624884s
Jan 25 06:51:56.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:51:58.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.063256016s
Jan 25 06:51:58.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:00.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.063744432s
Jan 25 06:52:00.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:02.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.063072578s
Jan 25 06:52:02.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:04.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.063303688s
Jan 25 06:52:04.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:06.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.063727664s
Jan 25 06:52:06.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:08.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.062258714s
Jan 25 06:52:08.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:10.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.062891926s
Jan 25 06:52:10.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:12.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.063350623s
Jan 25 06:52:12.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:14.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.062759645s
Jan 25 06:52:14.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:16.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.063712529s
Jan 25 06:52:16.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:18.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.062356152s
Jan 25 06:52:18.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:20.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.063344541s
Jan 25 06:52:20.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:22.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.062802716s
Jan 25 06:52:22.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:24.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.062572352s
Jan 25 06:52:24.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:26.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.062419906s
Jan 25 06:52:26.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:28.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.062996751s
Jan 25 06:52:28.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:30.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.063169906s
Jan 25 06:52:30.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:32.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.063248535s
Jan 25 06:52:32.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:34.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.062523572s
Jan 25 06:52:34.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:36.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.063580383s
Jan 25 06:52:36.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:38.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.062098941s
Jan 25 06:52:38.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:40.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.063535432s
Jan 25 06:52:40.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:42.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.062537881s
Jan 25 06:52:42.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:44.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.06302934s
Jan 25 06:52:44.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:46.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.06323007s
Jan 25 06:52:46.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:48.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.062777684s
Jan 25 06:52:48.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:50.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.063371825s
Jan 25 06:52:50.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:52.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.063347145s
Jan 25 06:52:52.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:54.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.062689789s
Jan 25 06:52:54.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:56.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.062454351s
Jan 25 06:52:56.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:52:58.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.0627253s
Jan 25 06:52:58.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:00.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.062331851s
Jan 25 06:53:00.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:02.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.063699303s
Jan 25 06:53:02.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:04.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.06277176s
Jan 25 06:53:04.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:06.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.062635259s
Jan 25 06:53:06.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:08.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.062975315s
Jan 25 06:53:08.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:10.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.063289344s
Jan 25 06:53:10.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:12.358: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.06404907s
Jan 25 06:53:12.358: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:14.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.06299959s
Jan 25 06:53:14.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:16.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.062609646s
Jan 25 06:53:16.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:18.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.062895437s
Jan 25 06:53:18.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:20.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.0629945s
Jan 25 06:53:20.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:22.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.062626463s
Jan 25 06:53:22.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:24.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.063476029s
Jan 25 06:53:24.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:26.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.063535842s
Jan 25 06:53:26.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:28.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.062983616s
Jan 25 06:53:28.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:30.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.06288583s
Jan 25 06:53:30.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:32.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.062732748s
Jan 25 06:53:32.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:34.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.06349115s
Jan 25 06:53:34.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:36.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.062672143s
Jan 25 06:53:36.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:38.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.062153597s
Jan 25 06:53:38.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:40.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.063353782s
Jan 25 06:53:40.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:42.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.062542157s
Jan 25 06:53:42.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:44.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.06287152s
Jan 25 06:53:44.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:46.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.062375904s
Jan 25 06:53:46.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:48.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.062654537s
Jan 25 06:53:48.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:50.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.062779957s
Jan 25 06:53:50.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:52.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.062567458s
Jan 25 06:53:52.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:54.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.062903726s
Jan 25 06:53:54.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:56.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.06241454s
Jan 25 06:53:56.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:53:58.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.063817114s
Jan 25 06:53:58.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:00.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.062619932s
Jan 25 06:54:00.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:02.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.062493016s
Jan 25 06:54:02.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:04.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.062454193s
Jan 25 06:54:04.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:06.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.06267528s
Jan 25 06:54:06.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:08.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.063428721s
Jan 25 06:54:08.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:10.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.063406697s
Jan 25 06:54:10.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:12.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.062930915s
Jan 25 06:54:12.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:14.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.063351112s
Jan 25 06:54:14.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:16.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.0626065s
Jan 25 06:54:16.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:18.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.062225806s
Jan 25 06:54:18.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:20.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.063546744s
Jan 25 06:54:20.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:22.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.062770973s
Jan 25 06:54:22.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:24.357: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.063375931s
Jan 25 06:54:24.357: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:26.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.062667927s
Jan 25 06:54:26.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:28.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.062454288s
Jan 25 06:54:28.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:30.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.062606269s
Jan 25 06:54:30.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:32.356: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.062297959s
Jan 25 06:54:32.356: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:34.359: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.06541819s
Jan 25 06:54:34.359: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:34.390: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.096876074s
Jan 25 06:54:34.391: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 25 06:54:34.391: FAIL: Pod did not start running: timed out while waiting for pod port-forwarding-5701/pfpod to be running and ready
Full Stack Trace
k8s.io/kubernetes/test/e2e/kubectl.doTestOverWebSockets({0x75c24b2, 0x7}, 0xc000c983c0)
test/e2e/kubectl/portforward.go:372 +0x2af
k8s.io/kubernetes/test/e2e/kubectl.glob..func2.1.3()
test/e2e/kubectl/portforward.go:471 +0x29
[AfterEach] [sig-cli] Kubectl Port forwarding
test/e2e/framework/node/init/init.go:32
Jan 25 06:54:34.391: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/25/23 06:54:34.427�[0m
�[1mSTEP:�[0m Collecting events from namespace "port-forwarding-5701". �[38;5;243m01/25/23 06:54:34.427�[0m
�[1mSTEP:�[0m Found 11 events. �[38;5;243m01/25/23 06:54:34.46�[0m
Jan 25 06:54:34.460: INFO: At 2023-01-25 06:49:34 +0000 UTC - event for pfpod: {default-scheduler } Scheduled: Successfully assigned port-forwarding-5701/pfpod to i-0df245ff74f46fa48
Jan 25 06:54:34.460: INFO: At 2023-01-25 06:49:34 +0000 UTC - event for pfpod: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "125e248322aa173b54df7ddc2b491ac8ffee11e3621248a89852997f6f45fecd": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:34.460: INFO: At 2023-01-25 06:49:48 +0000 UTC - event for pfpod: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a9851c8bd2ac9c11bcca8b290de48aa408a5fcdfffd06c79a89b9044f82b0108": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:34.460: INFO: At 2023-01-25 06:49:59 +0000 UTC - event for pfpod: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "32aa726cf8dfed914aa596ce655061c2534bbc076e9ca65948a9b70276d5addc": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:34.460: INFO: At 2023-01-25 06:50:12 +0000 UTC - event for pfpod: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "096f4af54132653de7b595155423ecb9bac1d0ab20bdf98fb0444838c56a1a56": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:34.460: INFO: At 2023-01-25 06:50:23 +0000 UTC - event for pfpod: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ec3156e80eff351a53be8b2c2049b6a36268da44d217ba584ca1d09c493127d4": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:34.460: INFO: At 2023-01-25 06:50:35 +0000 UTC - event for pfpod: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7f39302122c7f4596529e1a12d2f43e8e4524bc5172ea61c9222127f86059d43": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:34.460: INFO: At 2023-01-25 06:50:47 +0000 UTC - event for pfpod: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0572441a18b4efa9cc26ccdc9233fde6eaa6592d130bf00aef84995a619255db": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:34.460: INFO: At 2023-01-25 06:51:01 +0000 UTC - event for pfpod: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c53977e3df5a199b8ddf43f0755ee2617c5f5357b808b285d412b4e9eccb5409": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:34.460: INFO: At 2023-01-25 06:51:16 +0000 UTC - event for pfpod: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "47372476b081c314c44c524827a78b41b82cee2df005fc7f7d6add58203309a2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:34.460: INFO: At 2023-01-25 06:51:31 +0000 UTC - event for pfpod: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "01597f63bd9a31824206299da4a68087bfcfd201a6cd14dc92a7b52b7ccb69b1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:54:34.492: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 25 06:54:34.492: INFO: pfpod i-0df245ff74f46fa48 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:49:34 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:49:34 +0000 UTC ContainersNotReady containers with unready status: [readiness portforwardtester]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:49:34 +0000 UTC ContainersNotReady containers with unready status: [readiness portforwardtester]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:49:34 +0000 UTC }]
Jan 25 06:54:34.492: INFO:
Jan 25 06:54:34.525: INFO: Unable to fetch port-forwarding-5701/pfpod/readiness logs: the server rejected our request for an unknown reason (get pods pfpod)
Jan 25 06:54:34.559: INFO: Unable to fetch port-forwarding-5701/pfpod/portforwardtester logs: the server rejected our request for an unknown reason (get pods pfpod)
Jan 25 06:54:34.591: INFO:
Logging node info for node i-071cd767a01ed5a98
Jan 25 06:54:34.622: INFO: Node Info: &Node{ObjectMeta:{i-071cd767a01ed5a98 e176a57e-e197-4621-9ff9-1551f1909075 49349 0 2023-01-25 06:25:20 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-071cd767a01ed5a98 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ca-central-1a topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.39.108 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-071cd767a01ed5a98"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-25 06:25:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-25 06:25:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:26:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:26:12 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-25 06:26:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-25 06:53:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-071cd767a01ed5a98,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862937600 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758080000 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:26:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.108,},NodeAddress{Type:ExternalIP,Address:3.97.7.38,},NodeAddress{Type:InternalDNS,Address:i-071cd767a01ed5a98.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-071cd767a01ed5a98.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-97-7-38.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dc6c1622927717fff709d069d68b2,SystemUUID:ec2dc6c1-6229-2771-7fff-709d069d68b2,BootID:034bcfe8-9462-416d-9a80-bb9dcb9fd35d,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.1],SizeBytes:135178704,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.1],SizeBytes:124995897,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:63920476,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.1],SizeBytes:57661752,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:44339025,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42841295,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4965790,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 25 06:54:34.623: INFO:
Logging kubelet events for node i-071cd767a01ed5a98
Jan 25 06:54:34.658: INFO:
Logging pods the kubelet thinks is on node i-071cd767a01ed5a98
Jan 25 06:54:34.703: INFO: kube-scheduler-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:34.703: INFO: Container kube-scheduler ready: true, restart count 0
Jan 25 06:54:34.703: INFO: cilium-nqkjx started at 2023-01-25 06:26:00 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:54:34.703: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:54:34.703: INFO: dns-controller-69987775c6-7nx57 started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:34.703: INFO: Container dns-controller ready: true, restart count 0
Jan 25 06:54:34.703: INFO: aws-cloud-controller-manager-6kpfv started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:34.703: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 25 06:54:34.703: INFO: etcd-manager-events-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (10+1 container statuses recorded)
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Container etcd-manager ready: true, restart count 0
Jan 25 06:54:34.703: INFO: etcd-manager-main-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (10+1 container statuses recorded)
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Container etcd-manager ready: true, restart count 0
Jan 25 06:54:34.703: INFO: kube-apiserver-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+2 container statuses recorded)
Jan 25 06:54:34.703: INFO: Container healthcheck ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Container kube-apiserver ready: true, restart count 3
Jan 25 06:54:34.703: INFO: kube-controller-manager-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:34.703: INFO: Container kube-controller-manager ready: true, restart count 4
Jan 25 06:54:34.703: INFO: kops-controller-qfzp7 started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:34.703: INFO: Container kops-controller ready: true, restart count 0
Jan 25 06:54:34.703: INFO: ebs-csi-controller-5bd98b456f-cdjzf started at 2023-01-25 06:26:00 +0000 UTC (0+5 container statuses recorded)
Jan 25 06:54:34.703: INFO: Container csi-attacher ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Container csi-resizer ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:54:34.703: INFO: ebs-csi-node-7t2hw started at 2023-01-25 06:26:00 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:34.703: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:54:34.703: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:54:34.703: INFO: cilium-operator-5dd44dc49f-jhvqd started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:34.703: INFO: Container cilium-operator ready: true, restart count 0
Jan 25 06:54:34.963: INFO:
Latency metrics for node i-071cd767a01ed5a98
Jan 25 06:54:34.963: INFO:
Logging node info for node i-0dbcb4dc9e1f5e62b
Jan 25 06:54:34.994: INFO: Node Info: &Node{ObjectMeta:{i-0dbcb4dc9e1f5e62b f1ad0cae-2ed9-41a4-8be5-9884ae32adaa 49600 0 2023-01-25 06:27:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0dbcb4dc9e1f5e62b kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0dbcb4dc9e1f5e62b topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.63.11 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0dbcb4dc9e1f5e62b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:52:16 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:52:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0dbcb4dc9e1f5e62b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:38 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.11,},NodeAddress{Type:ExternalIP,Address:3.96.138.144,},NodeAddress{Type:InternalDNS,Address:i-0dbcb4dc9e1f5e62b.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0dbcb4dc9e1f5e62b.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-138-144.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2cc8535b4596aa012ef7857efd2302,SystemUUID:ec2cc853-5b45-96aa-012e-f7857efd2302,BootID:1d85d5b3-ad1c-462b-98e2-f01921a4e990,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-08b8d4339aaccaee6],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-08b8d4339aaccaee6,DevicePath:,},},Config:nil,},}
Jan 25 06:54:34.994: INFO:
Logging kubelet events for node i-0dbcb4dc9e1f5e62b
Jan 25 06:54:35.032: INFO:
Logging pods the kubelet thinks is on node i-0dbcb4dc9e1f5e62b
Jan 25 06:54:35.072: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-4wfp7 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.072: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.072: INFO: csi-mockplugin-resizer-0 started at 2023-01-25 06:50:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container csi-resizer ready: true, restart count 0
Jan 25 06:54:35.072: INFO: webserver-deployment-7f5969cbc7-pzv6w started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container httpd ready: false, restart count 0
Jan 25 06:54:35.072: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-psvwb started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.072: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.072: INFO: hostexec-i-0dbcb4dc9e1f5e62b-97cc8 started at 2023-01-25 06:50:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container agnhost-container ready: true, restart count 0
Jan 25 06:54:35.072: INFO: pod-subpath-test-dynamicpv-gwtk started at 2023-01-25 06:51:51 +0000 UTC (2+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Init container init-volume-dynamicpv-gwtk ready: false, restart count 0
Jan 25 06:54:35.072: INFO: Init container test-init-volume-dynamicpv-gwtk ready: false, restart count 0
Jan 25 06:54:35.072: INFO: Container test-container-subpath-dynamicpv-gwtk ready: false, restart count 0
Jan 25 06:54:35.072: INFO: ebs-csi-node-bn5vq started at 2023-01-25 06:27:26 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:54:35.072: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:54:35.072: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:54:35.072: INFO: busybox-1b7825ea-2c0f-4fcb-b7f9-c97bd78ff608 started at 2023-01-25 06:50:33 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container busybox ready: false, restart count 0
Jan 25 06:54:35.072: INFO: webserver-deployment-7f5969cbc7-ctzws started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container httpd ready: false, restart count 0
Jan 25 06:54:35.072: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.072: INFO: cilium-wptjp started at 2023-01-25 06:27:26 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:54:35.072: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:54:35.072: INFO: local-injector started at 2023-01-25 06:50:06 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container local-injector ready: false, restart count 0
Jan 25 06:54:35.072: INFO: pod-d973dfcb-fb6f-420f-9eb8-855583292025 started at 2023-01-25 06:51:32 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container test-container ready: false, restart count 0
Jan 25 06:54:35.072: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-8h8sv started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.072: INFO: webserver-deployment-7f5969cbc7-fqq2n started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container httpd ready: true, restart count 0
Jan 25 06:54:35.072: INFO: csi-mockplugin-attacher-0 started at 2023-01-25 06:50:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container csi-attacher ready: false, restart count 0
Jan 25 06:54:35.072: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rt5t5 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.072: INFO: csi-mockplugin-0 started at 2023-01-25 06:50:15 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:35.072: INFO: Container csi-provisioner ready: false, restart count 0
Jan 25 06:54:35.072: INFO: Container driver-registrar ready: false, restart count 0
Jan 25 06:54:35.072: INFO: Container mock ready: false, restart count 0
Jan 25 06:54:35.278: INFO:
Latency metrics for node i-0dbcb4dc9e1f5e62b
Jan 25 06:54:35.278: INFO:
Logging node info for node i-0df245ff74f46fa48
Jan 25 06:54:35.310: INFO: Node Info: &Node{ObjectMeta:{i-0df245ff74f46fa48 3893284f-3580-49f6-b7a1-2ddba9ae06ec 49602 0 2023-01-25 06:27:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0df245ff74f46fa48 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0df245ff74f46fa48 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.41.87 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-6879":"i-0df245ff74f46fa48","ebs.csi.aws.com":"i-0df245ff74f46fa48"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kops-controller Update v1 2023-01-25 06:27:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:50:23 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:50:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0df245ff74f46fa48,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:40 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.87,},NodeAddress{Type:ExternalIP,Address:3.96.184.142,},NodeAddress{Type:InternalDNS,Address:i-0df245ff74f46fa48.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0df245ff74f46fa48.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-184-142.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2129fadd2b2b52f98b14d7c0337fff,SystemUUID:ec2129fa-dd2b-2b52-f98b-14d7c0337fff,BootID:6b9e799a-3b2a-44dc-8a4e-1a86f3b5230b,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-020105115fe204770],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-020105115fe204770,DevicePath:,},},Config:nil,},}
Jan 25 06:54:35.311: INFO:
Logging kubelet events for node i-0df245ff74f46fa48
Jan 25 06:54:35.344: INFO:
Logging pods the kubelet thinks is on node i-0df245ff74f46fa48
Jan 25 06:54:35.382: INFO: ebs-csi-node-cfszw started at 2023-01-25 06:27:28 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:54:35.382: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:54:35.382: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:54:35.382: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x5hz9 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.382: INFO: pvc-volume-tester-mrm72 started at 2023-01-25 06:52:23 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container volume-tester ready: false, restart count 0
Jan 25 06:54:35.382: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.382: INFO: pfpod started at 2023-01-25 06:49:34 +0000 UTC (0+2 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container portforwardtester ready: false, restart count 0
Jan 25 06:54:35.382: INFO: Container readiness ready: false, restart count 0
Jan 25 06:54:35.382: INFO: csi-mockplugin-0 started at 2023-01-25 06:46:38 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:54:35.382: INFO: Container driver-registrar ready: true, restart count 0
Jan 25 06:54:35.382: INFO: Container mock ready: true, restart count 0
Jan 25 06:54:35.382: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-569r5 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.382: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-plznd started at 2023-01-25 06:46:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.382: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.382: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2mwzt started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.382: INFO: webserver-deployment-7f5969cbc7-q9lv4 started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container httpd ready: false, restart count 0
Jan 25 06:54:35.382: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hd52p started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.382: INFO: webserver-deployment-7f5969cbc7-hrp6g started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container httpd ready: false, restart count 0
Jan 25 06:54:35.382: INFO: webserver-deployment-7f5969cbc7-r4gfp started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container httpd ready: true, restart count 0
Jan 25 06:54:35.382: INFO: cilium-4kl5t started at 2023-01-25 06:27:28 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:54:35.382: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:54:35.382: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:54:35.382: INFO: pod-b2d5aa59-ddb2-497b-b629-98cd25649a85 started at 2023-01-25 06:50:17 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container write-pod ready: false, restart count 0
Jan 25 06:54:35.382: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-szf5f started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.382: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.554: INFO:
Latency metrics for node i-0df245ff74f46fa48
Jan 25 06:54:35.554: INFO:
Logging node info for node i-0e8434bc9e1236fd5
Jan 25 06:54:35.586: INFO: Node Info: &Node{ObjectMeta:{i-0e8434bc9e1236fd5 48f670c2-fe1c-46d3-b197-464836dab89d 49604 0 2023-01-25 06:27:23 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0e8434bc9e1236fd5 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0e8434bc9e1236fd5 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.54.30 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-5972":"i-0e8434bc9e1236fd5","ebs.csi.aws.com":"i-0e8434bc9e1236fd5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:50:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-25 06:52:43 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0e8434bc9e1236fd5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:35 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.30,},NodeAddress{Type:ExternalIP,Address:99.79.58.56,},NodeAddress{Type:InternalDNS,Address:i-0e8434bc9e1236fd5.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0e8434bc9e1236fd5.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-99-79-58-56.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2439e74c65fcfb2f8c6ece88db3944,SystemUUID:ec2439e7-4c65-fcfb-2f8c-6ece88db3944,BootID:568b3843-a4d7-46ef-b502-19f3e8310af2,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-5972^782f81a1-9c7c-11ed-97c4-deff35c7067a kubernetes.io/csi/ebs.csi.aws.com^vol-058ab77a5848fba8a kubernetes.io/csi/ebs.csi.aws.com^vol-06d3e55226f87a3e7],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-06d3e55226f87a3e7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-058ab77a5848fba8a,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-5972^782f81a1-9c7c-11ed-97c4-deff35c7067a,DevicePath:,},},Config:nil,},}
Jan 25 06:54:35.586: INFO:
Logging kubelet events for node i-0e8434bc9e1236fd5
Jan 25 06:54:35.620: INFO:
Logging pods the kubelet thinks is on node i-0e8434bc9e1236fd5
Jan 25 06:54:35.664: INFO: csi-mockplugin-attacher-0 started at 2023-01-25 06:49:32 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container csi-attacher ready: false, restart count 0
Jan 25 06:54:35.664: INFO: csi-hostpathplugin-0 started at 2023-01-25 06:49:50 +0000 UTC (0+7 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container csi-attacher ready: true, restart count 0
Jan 25 06:54:35.664: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:54:35.664: INFO: Container csi-resizer ready: true, restart count 0
Jan 25 06:54:35.664: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 25 06:54:35.664: INFO: Container hostpath ready: true, restart count 0
Jan 25 06:54:35.664: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:54:35.664: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:54:35.664: INFO: webserver-deployment-7f5969cbc7-t7ck4 started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container httpd ready: false, restart count 0
Jan 25 06:54:35.664: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sr5k2 started at 2023-01-25 06:46:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.664: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rbghp started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.664: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.664: INFO: pod-subpath-test-dynamicpv-mf6j started at 2023-01-25 06:49:53 +0000 UTC (0+2 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container test-container-subpath-dynamicpv-mf6j ready: false, restart count 0
Jan 25 06:54:35.664: INFO: Container test-container-volume-dynamicpv-mf6j ready: false, restart count 0
Jan 25 06:54:35.664: INFO: cilium-xkk5h started at 2023-01-25 06:27:24 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:54:35.664: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:54:35.664: INFO: inline-volume-tester-2shhw started at 2023-01-25 06:47:57 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 25 06:54:35.664: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2kf7l started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.664: INFO: pod-subpath-test-preprovisionedpv-vfbk started at 2023-01-25 06:51:06 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Init container init-volume-preprovisionedpv-vfbk ready: false, restart count 0
Jan 25 06:54:35.664: INFO: Container test-container-subpath-preprovisionedpv-vfbk ready: false, restart count 0
Jan 25 06:54:35.664: INFO: coredns-559769c974-ggjhw started at 2023-01-25 06:27:35 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container coredns ready: true, restart count 0
Jan 25 06:54:35.664: INFO: hostexec-i-0e8434bc9e1236fd5-9ql96 started at 2023-01-25 06:50:49 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container agnhost-container ready: true, restart count 0
Jan 25 06:54:35.664: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5mlr8 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.664: INFO: csi-mockplugin-0 started at 2023-01-25 06:49:32 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container csi-provisioner ready: false, restart count 0
Jan 25 06:54:35.664: INFO: Container driver-registrar ready: false, restart count 0
Jan 25 06:54:35.664: INFO: Container mock ready: false, restart count 0
Jan 25 06:54:35.664: INFO: hostexec-i-0e8434bc9e1236fd5-274ww started at 2023-01-25 06:50:47 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container agnhost-container ready: true, restart count 0
Jan 25 06:54:35.664: INFO: pod-9c25960d-410f-429b-b861-ab98bb2eb98b started at 2023-01-25 06:50:50 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container write-pod ready: false, restart count 0
Jan 25 06:54:35.664: INFO: webserver-deployment-7f5969cbc7-wwshs started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container httpd ready: false, restart count 0
Jan 25 06:54:35.664: INFO: ebs-csi-node-4d5tv started at 2023-01-25 06:27:24 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:54:35.664: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:54:35.664: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:54:35.664: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qqns6 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.664: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-lw7tl started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.664: INFO: coredns-autoscaler-7cb5c5b969-mndhj started at 2023-01-25 06:27:35 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container autoscaler ready: true, restart count 0
Jan 25 06:54:35.664: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-tczwz started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:35.664: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:35.900: INFO:
Latency metrics for node i-0e8434bc9e1236fd5
Jan 25 06:54:35.900: INFO:
Logging node info for node i-0ebbe4bd7ec58d213
Jan 25 06:54:35.931: INFO: Node Info: &Node{ObjectMeta:{i-0ebbe4bd7ec58d213 f69d5727-b51d-4595-ab9a-fe57b5c9c863 49639 0 2023-01-25 06:27:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0ebbe4bd7ec58d213 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0ebbe4bd7ec58d213 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.215 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0ebbe4bd7ec58d213"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:54:31 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:54:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0ebbe4bd7ec58d213,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051685376 0} {<nil>} 3956724Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946827776 0} {<nil>} 3854324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:54:31 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:54:31 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:54:31 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:54:31 +0000 UTC,LastTransitionTime:2023-01-25 06:27:38 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.215,},NodeAddress{Type:ExternalIP,Address:3.96.126.35,},NodeAddress{Type:InternalDNS,Address:i-0ebbe4bd7ec58d213.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ebbe4bd7ec58d213.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-126-35.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2da1db9b923c887fdbe1dd4557a0d5,SystemUUID:ec2da1db-9b92-3c88-7fdb-e1dd4557a0d5,BootID:10184fa3-2e15-48fe-9718-382b9d68823e,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0d005d0140f2dadfb],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d005d0140f2dadfb,DevicePath:,},},Config:nil,},}
Jan 25 06:54:35.932: INFO:
Logging kubelet events for node i-0ebbe4bd7ec58d213
Jan 25 06:54:35.966: INFO:
Logging pods the kubelet thinks is on node i-0ebbe4bd7ec58d213
Jan 25 06:54:36.009: INFO: pod-disruption-failure-ignore-0-l6qmf started at 2023-01-25 06:46:49 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container c ready: true, restart count 0
Jan 25 06:54:36.009: INFO: pod-disruption-failure-ignore-1-f77bm started at 2023-01-25 06:46:49 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container c ready: false, restart count 0
Jan 25 06:54:36.009: INFO: cilium-pkfh6 started at 2023-01-25 06:27:27 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:54:36.009: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:54:36.009: INFO: coredns-559769c974-v87jf started at 2023-01-25 06:27:57 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container coredns ready: true, restart count 0
Jan 25 06:54:36.009: INFO: inline-volume-tester-bns44 started at 2023-01-25 06:49:50 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 25 06:54:36.009: INFO: webserver-deployment-7f5969cbc7-wch65 started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container httpd ready: false, restart count 0
Jan 25 06:54:36.009: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-npbz4 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:36.009: INFO: pod-disruption-failure-ignore-1-vmrs8 started at 2023-01-25 06:47:22 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container c ready: false, restart count 0
Jan 25 06:54:36.009: INFO: csi-hostpathplugin-0 started at 2023-01-25 06:49:47 +0000 UTC (0+7 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container csi-attacher ready: false, restart count 0
Jan 25 06:54:36.009: INFO: Container csi-provisioner ready: false, restart count 0
Jan 25 06:54:36.009: INFO: Container csi-resizer ready: false, restart count 0
Jan 25 06:54:36.009: INFO: Container csi-snapshotter ready: false, restart count 0
Jan 25 06:54:36.009: INFO: Container hostpath ready: false, restart count 0
Jan 25 06:54:36.009: INFO: Container liveness-probe ready: false, restart count 0
Jan 25 06:54:36.009: INFO: Container node-driver-registrar ready: false, restart count 0
Jan 25 06:54:36.009: INFO: pod-7dc5023a-bc51-44e0-a5b1-2ac8cbb4d9a7 started at 2023-01-25 06:51:20 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container write-pod ready: false, restart count 0
Jan 25 06:54:36.009: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-wm7lb started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:36.009: INFO: pod-77e2c30c-da83-47c3-846b-f0c680a63e2d started at 2023-01-25 06:49:59 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container write-pod ready: false, restart count 0
Jan 25 06:54:36.009: INFO: webserver-deployment-7f5969cbc7-wg5sj started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container httpd ready: false, restart count 0
Jan 25 06:54:36.009: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x4pht started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:36.009: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ft4g2 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:36.009: INFO: hostexec-i-0ebbe4bd7ec58d213-9jht8 started at 2023-01-25 06:49:56 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container agnhost-container ready: true, restart count 0
Jan 25 06:54:36.009: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hqhjx started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:36.009: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-csrp6 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:36.009: INFO: hostexec-i-0ebbe4bd7ec58d213-brbmd started at 2023-01-25 06:51:14 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container agnhost-container ready: true, restart count 0
Jan 25 06:54:36.009: INFO: ebs-csi-node-9ngsq started at 2023-01-25 06:27:27 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:54:36.009: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:54:36.009: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:54:36.009: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bpq52 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:54:36.009: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:54:36.203: INFO:
Latency metrics for node i-0ebbe4bd7ec58d213
[DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "port-forwarding-5701" for this suite. �[38;5;243m01/25/23 06:54:36.203�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-node\]\skubelet\sClean\sup\spods\son\snode\skubelet\sshould\sbe\sable\sto\sdelete\s10\spods\sper\snode\sin\s1m0s\.$'
test/e2e/node/kubelet.go:359 k8s.io/kubernetes/test/e2e/node.glob..func5.2.3() test/e2e/node/kubelet.go:359 +0x390from junit_01.xml
[BeforeEach] [sig-node] kubelet set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/25/23 06:46:14.738�[0m Jan 25 06:46:14.739: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename kubelet �[38;5;243m01/25/23 06:46:14.739�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/25/23 06:46:14.845�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/25/23 06:46:14.907�[0m [BeforeEach] [sig-node] kubelet test/e2e/framework/metrics/init/init.go:31 [BeforeEach] [sig-node] kubelet test/e2e/node/kubelet.go:278 [BeforeEach] Clean up pods on node test/e2e/node/kubelet.go:299 [It] kubelet should be able to delete 10 pods per node in 1m0s. test/e2e/node/kubelet.go:346 �[1mSTEP:�[0m Creating a RC of 40 pods and wait until all pods of this RC are running �[38;5;243m01/25/23 06:46:15.332�[0m �[1mSTEP:�[0m creating replication controller cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 in namespace kubelet-9040 �[38;5;243m01/25/23 06:46:15.332�[0m I0125 06:46:15.376372 6778 runners.go:193] Created replication controller with name: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8, namespace: kubelet-9040, replica count: 40 Jan 25 06:46:15.451: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:46:15.476: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:46:15.488: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:46:15.910: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:46:15.935: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:46:20.562: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:46:20.563: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:46:20.605: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:46:21.016: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:46:21.070: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" I0125 06:46:25.428093 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 10 running, 30 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:46:25.627: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:46:25.661: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:46:25.710: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:46:26.127: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:46:26.157: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:46:30.724: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:46:30.810: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:46:30.936: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:46:31.271: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:46:31.316: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" I0125 06:46:35.428896 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 25 running, 15 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:46:35.803: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:46:35.912: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:46:36.245: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:46:36.344: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:46:36.443: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:46:40.869: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:46:41.060: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:46:41.370: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:46:41.439: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:46:41.526: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" I0125 06:46:45.430119 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 27 running, 13 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:46:45.935: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:46:46.180: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:46:46.493: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:46:46.512: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:46:46.600: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:46:51.008: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:46:51.366: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:46:51.601: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:46:51.623: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:46:51.668: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" I0125 06:46:55.430727 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 29 running, 11 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:46:56.104: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:46:56.471: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:46:56.674: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:46:56.721: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:46:56.766: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:47:01.183: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:47:01.737: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:47:01.769: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:47:01.855: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:47:01.884: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" I0125 06:47:05.431791 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 31 running, 9 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:47:06.247: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:47:06.836: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:47:06.841: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:47:06.964: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:47:07.038: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:47:11.307: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:47:11.946: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:47:11.951: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:47:12.056: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:47:12.234: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" I0125 06:47:15.432584 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 32 running, 8 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:47:16.372: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:47:17.019: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:47:17.041: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:47:17.261: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:47:17.399: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:47:21.461: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:47:22.085: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:47:22.134: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:47:22.341: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:47:22.481: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" I0125 06:47:25.433816 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 32 running, 8 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:47:26.527: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:47:27.160: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:47:27.337: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:47:27.451: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:47:27.587: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:47:31.604: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:47:32.240: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:47:32.518: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:47:32.536: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:47:32.682: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" I0125 06:47:35.435009 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 32 running, 8 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:47:36.675: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:47:37.308: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:47:37.639: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:47:37.769: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:47:37.803: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:47:41.740: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:47:42.403: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:47:42.729: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:47:42.877: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:47:42.947: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" I0125 06:47:45.435757 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 33 running, 7 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:47:46.810: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:47:47.481: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:47:47.804: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:47:48.003: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:47:48.046: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:47:51.884: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:47:52.549: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:47:52.889: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:47:53.160: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:47:53.221: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" I0125 06:47:55.436379 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 34 running, 6 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:47:56.948: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:47:57.629: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:47:57.966: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:47:58.308: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:47:58.364: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:48:02.022: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:48:02.697: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:48:03.059: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:48:03.422: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:48:03.558: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" I0125 06:48:05.436640 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 36 running, 4 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:48:07.106: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:48:07.771: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:48:08.144: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:48:08.600: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:48:08.692: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:48:12.171: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:48:12.849: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:48:13.229: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:48:13.780: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:48:13.832: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" I0125 06:48:15.436912 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 36 running, 4 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:48:17.266: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:48:17.933: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:48:18.322: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:48:18.853: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:48:18.932: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:48:22.360: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:48:23.035: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:48:23.414: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:48:24.033: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:48:24.086: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" I0125 06:48:25.437974 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 36 running, 4 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:48:27.478: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:48:28.104: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:48:28.486: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:48:29.161: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:48:29.229: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:48:32.541: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:48:33.214: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:48:33.619: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:48:34.241: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:48:34.322: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" I0125 06:48:35.438730 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 37 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:48:37.606: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:48:38.297: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:48:38.705: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:48:39.364: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:48:39.429: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:48:42.673: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:48:43.378: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:48:43.785: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:48:44.438: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:48:44.630: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" I0125 06:48:45.439958 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 38 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:48:47.739: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:48:48.454: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:48:48.875: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:48:49.646: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:48:49.731: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:48:52.809: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:48:53.530: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:48:53.985: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:48:54.735: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:48:54.818: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" I0125 06:48:55.440710 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 38 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:48:57.877: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:48:58.600: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:48:59.110: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:48:59.815: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:48:59.914: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:49:02.951: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:49:03.685: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:49:04.269: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:49:04.888: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:49:05.270: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" I0125 06:49:05.441703 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 38 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:49:08.019: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:49:08.781: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:49:09.370: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:49:09.959: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:49:10.361: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:49:13.118: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:49:13.914: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:49:14.437: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:49:15.046: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" I0125 06:49:15.442439 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 38 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:49:15.457: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:49:18.199: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:49:19.007: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:49:19.513: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:49:20.156: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:49:20.595: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:49:23.278: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:49:24.089: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:49:24.590: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:49:25.242: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" I0125 06:49:25.442726 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:49:25.753: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:49:28.349: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:49:29.159: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:49:29.690: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:49:30.328: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:49:30.962: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:49:33.414: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:49:34.239: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:49:34.761: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:49:35.408: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" I0125 06:49:35.443323 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:49:36.079: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:49:38.480: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:49:39.318: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:49:39.867: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:49:40.483: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:49:41.158: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:49:43.596: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:49:44.386: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:49:44.958: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" I0125 06:49:45.443642 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:49:45.644: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:49:46.536: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:49:48.674: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:49:49.458: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:49:50.029: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:49:50.759: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:49:51.640: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:49:53.790: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:49:54.700: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:49:55.159: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" I0125 06:49:55.443930 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:49:55.855: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:49:56.738: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:49:58.856: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:49:59.792: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:50:00.329: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:50:01.121: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:50:02.224: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:50:03.922: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:50:04.879: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" I0125 06:50:05.444467 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:50:05.496: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:50:06.283: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:50:07.320: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:50:08.992: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:50:09.964: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:50:10.584: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:50:11.452: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:50:12.413: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:50:14.059: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:50:15.044: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" I0125 06:50:15.445455 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:50:15.661: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:50:16.590: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:50:17.543: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:50:19.126: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:50:20.134: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:50:20.870: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:50:21.666: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:50:22.621: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:50:24.192: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:50:25.220: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" I0125 06:50:25.446370 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:50:26.050: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:50:26.736: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:50:27.724: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:50:29.260: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:50:30.307: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:50:31.154: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:50:31.808: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:50:32.811: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:50:34.332: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:50:35.392: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" I0125 06:50:35.446873 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:50:36.241: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:50:36.987: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:50:37.899: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:50:39.403: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:50:40.493: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:50:41.339: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:50:42.064: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:50:42.976: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:50:44.473: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" I0125 06:50:45.447550 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:50:45.616: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:50:46.443: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:50:47.164: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:50:48.045: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:50:49.560: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:50:50.723: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:50:51.514: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:50:52.346: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:50:53.181: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:50:54.700: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" I0125 06:50:55.447766 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:50:55.860: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:50:56.592: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:50:57.573: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:50:58.249: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:50:59.775: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:51:00.995: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:51:02.002: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:51:02.684: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:51:03.788: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:51:04.839: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" I0125 06:51:05.448886 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:51:06.106: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:51:07.088: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:51:07.792: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:51:08.879: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:51:09.906: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:51:11.194: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:51:12.167: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:51:12.882: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:51:13.990: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:51:14.976: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" I0125 06:51:15.449517 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:51:16.333: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:51:17.274: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:51:18.164: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:51:19.199: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:51:20.072: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:51:21.424: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:51:22.394: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:51:23.261: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:51:24.295: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:51:25.140: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" I0125 06:51:25.450578 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:51:26.512: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:51:27.490: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:51:28.409: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:51:29.401: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:51:30.236: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:51:31.613: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:51:32.578: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:51:33.502: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:51:34.523: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:51:35.313: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" I0125 06:51:35.451814 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:51:36.710: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:51:37.655: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:51:38.612: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:51:39.626: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:51:40.382: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:51:41.927: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:51:42.788: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:51:43.692: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:51:44.698: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:51:45.448: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" I0125 06:51:45.452380 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:51:47.013: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:51:47.967: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:51:48.775: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:51:49.812: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:51:50.513: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:51:52.104: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:51:53.048: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:51:54.061: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:51:54.881: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" I0125 06:51:55.452964 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:51:55.592: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:51:57.196: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:51:58.208: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:51:59.141: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:51:59.963: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:52:00.661: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:52:02.276: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:52:03.419: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:52:04.282: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:52:05.094: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" I0125 06:52:05.454151 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:52:05.729: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:52:07.368: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:52:08.510: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:52:09.366: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:52:10.180: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:52:10.800: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:52:12.467: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:52:13.627: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:52:14.512: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:52:15.261: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" I0125 06:52:15.455190 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:52:15.868: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:52:17.553: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:52:18.703: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:52:19.606: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:52:20.342: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:52:20.973: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:52:22.647: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:52:23.980: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:52:24.731: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" I0125 06:52:25.455982 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:52:25.460: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:52:26.042: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:52:27.736: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:52:29.088: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:52:29.808: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:52:30.555: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:52:31.149: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:52:32.889: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:52:34.196: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:52:34.887: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" I0125 06:52:35.456753 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:52:35.676: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:52:36.218: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:52:37.984: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:52:39.425: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:52:39.969: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:52:40.780: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:52:41.283: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:52:43.076: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:52:44.541: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:52:45.154: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" I0125 06:52:45.457614 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:52:45.874: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:52:46.350: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:52:48.167: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:52:49.668: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:52:50.234: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:52:50.953: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:52:51.419: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:52:53.264: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:52:54.830: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:52:55.355: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" I0125 06:52:55.458126 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:52:56.059: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:52:56.489: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:52:58.351: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:52:59.929: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:53:00.433: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:53:01.136: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:53:01.558: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:53:03.477: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:53:05.143: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" I0125 06:53:05.458688 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:53:05.514: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:53:06.211: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:53:06.625: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:53:08.574: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:53:10.255: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:53:10.594: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:53:11.336: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:53:11.690: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:53:13.744: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:53:15.338: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" I0125 06:53:15.458919 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:53:15.684: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:53:16.428: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:53:16.760: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:53:18.850: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:53:20.418: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:53:20.860: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:53:21.506: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:53:21.832: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:53:23.996: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" I0125 06:53:25.459877 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:53:25.524: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:53:25.938: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:53:26.592: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:53:26.902: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:53:29.099: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:53:30.623: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:53:31.042: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:53:31.697: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:53:31.980: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:53:34.196: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" I0125 06:53:35.460765 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:53:35.723: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:53:36.135: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:53:36.777: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:53:37.059: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:53:39.303: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:53:40.810: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:53:41.218: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:53:41.872: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:53:42.126: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:53:44.407: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" I0125 06:53:45.461860 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:53:45.941: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:53:46.312: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:53:46.962: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:53:47.194: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:53:49.494: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:53:51.059: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:53:51.400: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:53:52.061: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:53:52.264: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:53:54.599: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" I0125 06:53:55.462426 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:53:56.145: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:53:56.472: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:53:57.155: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:53:57.331: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:53:59.693: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:54:01.279: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:54:01.855: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:54:02.239: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:54:02.395: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:54:04.781: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" I0125 06:54:05.462704 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:54:06.359: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:54:06.948: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:54:07.325: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:54:07.458: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:54:09.877: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:54:11.498: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:54:12.062: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:54:12.466: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:54:12.523: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:54:15.012: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" I0125 06:54:15.463634 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady Jan 25 06:54:16.628: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:54:17.185: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:54:17.581: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:54:17.604: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:54:20.107: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" Jan 25 06:54:21.702: INFO: Missing info/stats for container "runtime" on node "i-0ebbe4bd7ec58d213" Jan 25 06:54:22.366: INFO: Missing info/stats for container "runtime" on node "i-0dbcb4dc9e1f5e62b" Jan 25 06:54:22.680: INFO: Missing info/stats for container "runtime" on node "i-071cd767a01ed5a98" Jan 25 06:54:22.684: INFO: Missing info/stats for container "runtime" on node "i-0df245ff74f46fa48" Jan 25 06:54:25.279: INFO: Missing info/stats for container "runtime" on node "i-0e8434bc9e1236fd5" I0125 06:54:25.464389 6778 runners.go:193] cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Pods: 40 out of 40 created, 39 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0125 06:54:25.499005 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2kf7l i-0e8434bc9e1236fd5 Running <nil> I0125 06:54:25.499065 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2mwzt i-0df245ff74f46fa48 Running <nil> I0125 06:54:25.499073 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-4wfp7 i-0dbcb4dc9e1f5e62b Running <nil> I0125 06:54:25.499080 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-569r5 i-0df245ff74f46fa48 Running <nil> I0125 06:54:25.499087 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h i-0dbcb4dc9e1f5e62b Running <nil> I0125 06:54:25.499093 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5mlr8 i-0e8434bc9e1236fd5 Running <nil> I0125 06:54:25.499100 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-7zg9l i-0ebbe4bd7ec58d213 Running <nil> I0125 06:54:25.499107 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-8h8sv i-0dbcb4dc9e1f5e62b Running <nil> I0125 06:54:25.499114 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9 i-0df245ff74f46fa48 Running <nil> I0125 06:54:25.499120 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bpq52 i-0ebbe4bd7ec58d213 Running <nil> I0125 06:54:25.499127 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-csrp6 i-0ebbe4bd7ec58d213 Running <nil> I0125 06:54:25.499140 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt i-0dbcb4dc9e1f5e62b Running <nil> I0125 06:54:25.499147 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ft4g2 i-0ebbe4bd7ec58d213 Running <nil> I0125 06:54:25.499154 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc i-0df245ff74f46fa48 Running <nil> I0125 06:54:25.499161 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h i-0dbcb4dc9e1f5e62b Running <nil> I0125 06:54:25.499168 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hd52p i-0df245ff74f46fa48 Running <nil> I0125 06:54:25.499174 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hqhjx i-0ebbe4bd7ec58d213 Running <nil> I0125 06:54:25.499181 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc i-0e8434bc9e1236fd5 Running <nil> I0125 06:54:25.499188 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jjlq8 i-0dbcb4dc9e1f5e62b Running <nil> I0125 06:54:25.499194 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jspbp i-0e8434bc9e1236fd5 Running <nil> I0125 06:54:25.499201 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-k9wft i-0ebbe4bd7ec58d213 Running <nil> I0125 06:54:25.499208 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-lw7tl i-0e8434bc9e1236fd5 Running <nil> I0125 06:54:25.499214 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-npbz4 i-0ebbe4bd7ec58d213 Running <nil> I0125 06:54:25.499223 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-plznd i-0df245ff74f46fa48 Running <nil> I0125 06:54:25.499230 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-psvwb i-0dbcb4dc9e1f5e62b Running <nil> I0125 06:54:25.499237 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj i-0dbcb4dc9e1f5e62b Running <nil> I0125 06:54:25.499244 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qqns6 i-0e8434bc9e1236fd5 Running <nil> I0125 06:54:25.499250 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qs6zz i-0ebbe4bd7ec58d213 Running <nil> I0125 06:54:25.499257 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rbghp i-0e8434bc9e1236fd5 Running <nil> I0125 06:54:25.499264 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rt5t5 i-0dbcb4dc9e1f5e62b Running <nil> I0125 06:54:25.499270 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rzvq9 i-0df245ff74f46fa48 Running <nil> I0125 06:54:25.499278 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sr5k2 i-0e8434bc9e1236fd5 Running <nil> I0125 06:54:25.499285 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sxzm4 i-0df245ff74f46fa48 Running <nil> I0125 06:54:25.499291 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-szf5f i-0df245ff74f46fa48 Running <nil> I0125 06:54:25.499298 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-tczwz i-0e8434bc9e1236fd5 Running <nil> I0125 06:54:25.499305 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-wm7lb i-0ebbe4bd7ec58d213 Running <nil> I0125 06:54:25.499312 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x4pht i-0ebbe4bd7ec58d213 Running <nil> I0125 06:54:25.499318 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x5hz9 i-0df245ff74f46fa48 Running <nil> I0125 06:54:25.499325 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xqz46 i-0dbcb4dc9e1f5e62b Running <nil> I0125 06:54:25.499331 6778 runners.go:193] Pod cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj i-0e8434bc9e1236fd5 Pending <nil> Jan 25 06:54:25.499: INFO: Unexpected error: <*errors.errorString | 0xc0013af570>: { s: "only 39 pods started out of 40", } Jan 25 06:54:25.499: FAIL: only 39 pods started out of 40 Full Stack Trace k8s.io/kubernetes/test/e2e/node.glob..func5.2.3() test/e2e/node/kubelet.go:359 +0x390 [AfterEach] Clean up pods on node test/e2e/node/kubelet.go:330 �[1mSTEP:�[0m removing the label kubelet_cleanup off the node i-0dbcb4dc9e1f5e62b �[38;5;243m01/25/23 06:54:25.499�[0m �[1mSTEP:�[0m verifying the node doesn't have the label kubelet_cleanup �[38;5;243m01/25/23 06:54:25.567�[0m �[1mSTEP:�[0m removing the label kubelet_cleanup off the node i-0df245ff74f46fa48 �[38;5;243m01/25/23 06:54:25.599�[0m �[1mSTEP:�[0m verifying the node doesn't have the label kubelet_cleanup �[38;5;243m01/25/23 06:54:25.668�[0m �[1mSTEP:�[0m removing the label kubelet_cleanup off the node i-0e8434bc9e1236fd5 �[38;5;243m01/25/23 06:54:25.7�[0m �[1mSTEP:�[0m verifying the node doesn't have the label kubelet_cleanup �[38;5;243m01/25/23 06:54:25.774�[0m �[1mSTEP:�[0m removing the label kubelet_cleanup off the node i-0ebbe4bd7ec58d213 �[38;5;243m01/25/23 06:54:25.806�[0m �[1mSTEP:�[0m verifying the node doesn't have the label kubelet_cleanup �[38;5;243m01/25/23 06:54:25.876�[0m [AfterEach] [sig-node] kubelet test/e2e/framework/node/init/init.go:32 Jan 25 06:54:25.909: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-node] kubelet test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-node] kubelet dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/25/23 06:54:25.941�[0m �[1mSTEP:�[0m Collecting events from namespace "kubelet-9040". �[38;5;243m01/25/23 06:54:25.941�[0m �[1mSTEP:�[0m Found 258 events. �[38;5;243m01/25/23 06:54:25.978�[0m Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8: {replication-controller } SuccessfulCreate: Created pod: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2kf7l Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8: {replication-controller } SuccessfulCreate: Created pod: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x5hz9 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8: {replication-controller } SuccessfulCreate: Created pod: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hqhjx Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8: {replication-controller } SuccessfulCreate: Created pod: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-psvwb Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8: {replication-controller } SuccessfulCreate: Created pod: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5mlr8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8: {replication-controller } SuccessfulCreate: Created pod: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hd52p Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8: {replication-controller } SuccessfulCreate: Created pod: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-k9wft Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8: {replication-controller } SuccessfulCreate: Created pod: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-4wfp7 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8: {replication-controller } SuccessfulCreate: Created pod: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2mwzt Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2kf7l: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2kf7l to i-0e8434bc9e1236fd5 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2mwzt: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2mwzt to i-0df245ff74f46fa48 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-4wfp7: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-4wfp7 to i-0dbcb4dc9e1f5e62b Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-569r5: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-569r5 to i-0df245ff74f46fa48 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h to i-0dbcb4dc9e1f5e62b Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5mlr8: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5mlr8 to i-0e8434bc9e1236fd5 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-8h8sv: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-8h8sv to i-0dbcb4dc9e1f5e62b Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9 to i-0df245ff74f46fa48 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bpq52: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bpq52 to i-0ebbe4bd7ec58d213 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-csrp6: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-csrp6 to i-0ebbe4bd7ec58d213 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt to i-0dbcb4dc9e1f5e62b Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ft4g2: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ft4g2 to i-0ebbe4bd7ec58d213 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc to i-0df245ff74f46fa48 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h to i-0dbcb4dc9e1f5e62b Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hd52p: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hd52p to i-0df245ff74f46fa48 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hqhjx: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hqhjx to i-0ebbe4bd7ec58d213 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc to i-0e8434bc9e1236fd5 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jjlq8: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jjlq8 to i-0dbcb4dc9e1f5e62b Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jspbp: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jspbp to i-0e8434bc9e1236fd5 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-k9wft: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-k9wft to i-0ebbe4bd7ec58d213 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-lw7tl: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-lw7tl to i-0e8434bc9e1236fd5 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-npbz4: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-npbz4 to i-0ebbe4bd7ec58d213 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-psvwb: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-psvwb to i-0dbcb4dc9e1f5e62b Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj to i-0dbcb4dc9e1f5e62b Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qqns6: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qqns6 to i-0e8434bc9e1236fd5 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qs6zz: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qs6zz to i-0ebbe4bd7ec58d213 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rbghp: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rbghp to i-0e8434bc9e1236fd5 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rt5t5: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rt5t5 to i-0dbcb4dc9e1f5e62b Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rzvq9: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rzvq9 to i-0df245ff74f46fa48 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sxzm4: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sxzm4 to i-0df245ff74f46fa48 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-szf5f: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-szf5f to i-0df245ff74f46fa48 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-tczwz: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-tczwz to i-0e8434bc9e1236fd5 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-wm7lb: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-wm7lb to i-0ebbe4bd7ec58d213 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x4pht: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x4pht to i-0ebbe4bd7ec58d213 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x5hz9: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x5hz9 to i-0df245ff74f46fa48 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xqz46: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xqz46 to i-0dbcb4dc9e1f5e62b Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj to i-0e8434bc9e1236fd5 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:16 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8: {replication-controller } SuccessfulCreate: (combined from similar events): Created pod: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:16 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-7zg9l: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-7zg9l to i-0ebbe4bd7ec58d213 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:16 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-plznd: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-plznd to i-0df245ff74f46fa48 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:16 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sr5k2: {default-scheduler } Scheduled: Successfully assigned kubelet-9040/cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sr5k2 to i-0e8434bc9e1236fd5 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2kf7l: {kubelet i-0e8434bc9e1236fd5} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2kf7l: {kubelet i-0e8434bc9e1236fd5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2mwzt: {kubelet i-0df245ff74f46fa48} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2mwzt: {kubelet i-0df245ff74f46fa48} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2mwzt: {kubelet i-0df245ff74f46fa48} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-4wfp7: {kubelet i-0dbcb4dc9e1f5e62b} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-4wfp7: {kubelet i-0dbcb4dc9e1f5e62b} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-4wfp7: {kubelet i-0dbcb4dc9e1f5e62b} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d9ac3da96b97eeb3ed94122a2a6a2f3fe82a26da07cfc21fa31affb7b2508fbc": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5mlr8: {kubelet i-0e8434bc9e1236fd5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-8h8sv: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bb145ecf74fc6a9852b4743d0a7aa4a3777f07ebd6e548e7611f3f29a998eb62": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bpq52: {kubelet i-0ebbe4bd7ec58d213} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bpq52: {kubelet i-0ebbe4bd7ec58d213} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bpq52: {kubelet i-0ebbe4bd7ec58d213} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-csrp6: {kubelet i-0ebbe4bd7ec58d213} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-csrp6: {kubelet i-0ebbe4bd7ec58d213} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "29711af241cdf5fbdba2330629487315a6b298d6776844077b1b6557ae704b3c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8a3fc11744ae894c0fece758a7453f6a495d2985dfb74ae5e39cf11d9d0ca745": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hd52p: {kubelet i-0df245ff74f46fa48} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jjlq8: {kubelet i-0dbcb4dc9e1f5e62b} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-psvwb: {kubelet i-0dbcb4dc9e1f5e62b} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qs6zz: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9b907c6434042f149f8359de766066663d91cbb37caf3ad745d019a5f921d378": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:17 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xqz46: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0a1b5c0ae781af31fd34ad406e906d2476b1c5bd477e71583f412ac0b81fb164": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2kf7l: {kubelet i-0e8434bc9e1236fd5} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-569r5: {kubelet i-0df245ff74f46fa48} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-569r5: {kubelet i-0df245ff74f46fa48} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-569r5: {kubelet i-0df245ff74f46fa48} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5mlr8: {kubelet i-0e8434bc9e1236fd5} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5mlr8: {kubelet i-0e8434bc9e1236fd5} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "888c9d1864078c62c06f9c2444b6e30bc3e0c16bfa1c6baf7c05111b275aa28d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-csrp6: {kubelet i-0ebbe4bd7ec58d213} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0769d8cc2d43a830cbefc7a9599cbb0f371fb09ae8f5c73ea043626e2a9fd984": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hd52p: {kubelet i-0df245ff74f46fa48} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hd52p: {kubelet i-0df245ff74f46fa48} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hqhjx: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1e5496d7e41dad83f853407657bd623962ef68f6775909feea93d756d4779094": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "66f84b425c27441c59f3fed651de42e632b9321cd83cc3a358d22388ad658fa6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jjlq8: {kubelet i-0dbcb4dc9e1f5e62b} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jjlq8: {kubelet i-0dbcb4dc9e1f5e62b} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jspbp: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "dbc38c0b6c181de6ca1e4d343602eff6dcb201f48260fb04646c8e45a1e0df5e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-k9wft: {kubelet i-0ebbe4bd7ec58d213} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-k9wft: {kubelet i-0ebbe4bd7ec58d213} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-k9wft: {kubelet i-0ebbe4bd7ec58d213} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-lw7tl: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d77ac295e3accf8844f80891d03b92ebf99703987b781ce5559f72727589d7ba": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-npbz4: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0ac33d10772065564cc6dbdeacafd0b8ae594f0ad4f10736690d18989ef903c3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-plznd: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "dbaccb01c94036f062123623aad6672c790636fa69e6ba18fbcd400218b1aee3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-psvwb: {kubelet i-0dbcb4dc9e1f5e62b} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-psvwb: {kubelet i-0dbcb4dc9e1f5e62b} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7df5a12a137741532f552d60521769cb32561832a49a80a7b3fee64d47116f91": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.978: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qqns6: {kubelet i-0e8434bc9e1236fd5} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qqns6: {kubelet i-0e8434bc9e1236fd5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qqns6: {kubelet i-0e8434bc9e1236fd5} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rbghp: {kubelet i-0e8434bc9e1236fd5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rbghp: {kubelet i-0e8434bc9e1236fd5} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rbghp: {kubelet i-0e8434bc9e1236fd5} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rt5t5: {kubelet i-0dbcb4dc9e1f5e62b} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rt5t5: {kubelet i-0dbcb4dc9e1f5e62b} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rt5t5: {kubelet i-0dbcb4dc9e1f5e62b} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rzvq9: {kubelet i-0df245ff74f46fa48} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rzvq9: {kubelet i-0df245ff74f46fa48} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rzvq9: {kubelet i-0df245ff74f46fa48} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sxzm4: {kubelet i-0df245ff74f46fa48} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sxzm4: {kubelet i-0df245ff74f46fa48} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sxzm4: {kubelet i-0df245ff74f46fa48} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-szf5f: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ff64ec9d5f5f2e496a99b9cdca154a4633a30f815536c1c91be438b9452880b6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-tczwz: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1efbbd5b8a8671766114691d64a26fb3f6902ae4bdc5e06c02f626e4ddcf6784": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-wm7lb: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4e7802a202a52437430758ad46a80682122f9007964f989ca5a2beb98b9f6676": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x5hz9: {kubelet i-0df245ff74f46fa48} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x5hz9: {kubelet i-0df245ff74f46fa48} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x5hz9: {kubelet i-0df245ff74f46fa48} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "49e79ba53513f5818dd6536b66ae29174e3bde529b164d74233046775bfe469f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:19 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-7zg9l: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "13191527c9d0aabae4625377e190efc53b57c95fbaaec7eb152f74963fcc5404": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:19 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ft4g2: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "afe0cd6b5d8676a802597e3dc7b1c25fc5c17d44916eaed4e7f408b49b89a288": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:19 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sr5k2: {kubelet i-0e8434bc9e1236fd5} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:19 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sr5k2: {kubelet i-0e8434bc9e1236fd5} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:19 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sr5k2: {kubelet i-0e8434bc9e1236fd5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:19 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x4pht: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bb8e7e0fa331cacf7b266b0a758db6b2647b70efb6826c1446c268cf3976ea05": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:28 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-8h8sv: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "79e9a048500da82fd6cf2901de7ab3b1bd1146a66a5bbc10bb896273631954e7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:29 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qs6zz: {kubelet i-0ebbe4bd7ec58d213} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:29 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qs6zz: {kubelet i-0ebbe4bd7ec58d213} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:29 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qs6zz: {kubelet i-0ebbe4bd7ec58d213} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:30 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-lw7tl: {kubelet i-0e8434bc9e1236fd5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:30 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-lw7tl: {kubelet i-0e8434bc9e1236fd5} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:30 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-npbz4: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "390ce27e426e7f934b9f97ca87c07aa127cb50e24f687e420c269bec4630b760": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:30 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-tczwz: {kubelet i-0e8434bc9e1236fd5} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:30 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-tczwz: {kubelet i-0e8434bc9e1236fd5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a43e2d3499345982d6394fd29aaebceba3d84141dd4bf83b6685b0e2527d1c1f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f3d387b92d08e734ec81a36d1676ec8e471b214e6ad3f47ba6eef6592112904e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hqhjx: {kubelet i-0ebbe4bd7ec58d213} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hqhjx: {kubelet i-0ebbe4bd7ec58d213} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hqhjx: {kubelet i-0ebbe4bd7ec58d213} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jspbp: {kubelet i-0e8434bc9e1236fd5} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jspbp: {kubelet i-0e8434bc9e1236fd5} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jspbp: {kubelet i-0e8434bc9e1236fd5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-lw7tl: {kubelet i-0e8434bc9e1236fd5} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-plznd: {kubelet i-0df245ff74f46fa48} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-plznd: {kubelet i-0df245ff74f46fa48} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-plznd: {kubelet i-0df245ff74f46fa48} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6c1484197c7d75f5f860e9525bb4f922c2568a9fc60f9942bdba9c7dd0c742e9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-tczwz: {kubelet i-0e8434bc9e1236fd5} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x4pht: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6fbd35ae37ef2d50e22ebccee58dd322234b66954594059d7ccee81449752d1b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xqz46: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4eba80bcde4484bb746c0c71986110485e3792dc372317e5b10c4c257cf67b73": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:31 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5ac8edaf91f2bfb085ab637fac0f26a7ce6be0ee9a578ac6633c33f53f545c7b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:32 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6d001bf08f4197d06147b2ce8b03ea5e74e77fc290e252cccaa902a37ed5027a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:32 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ef318d666dd7018662c6f2549600a4d548360abad16e641fe19135c8cf4f94a8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:32 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6a16fdbb99aeced95833d17e944fcc4ba3c2813b6c6571dd0cdf4fd3490213ba": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:32 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-wm7lb: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "299644d3674b8c3839815ca642c0cfe487dc6ecbcabfac6b428b4a19c436078f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:33 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-7zg9l: {kubelet i-0ebbe4bd7ec58d213} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:33 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-7zg9l: {kubelet i-0ebbe4bd7ec58d213} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:33 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-7zg9l: {kubelet i-0ebbe4bd7ec58d213} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:34 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f4077630d545de8d8c162260d9c15f0510ae064b1756120a47cd58de9baf5935": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:34 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ft4g2: {kubelet i-0ebbe4bd7ec58d213} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:34 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ft4g2: {kubelet i-0ebbe4bd7ec58d213} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:34 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ft4g2: {kubelet i-0ebbe4bd7ec58d213} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:34 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-szf5f: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8790c622830f9ace1c5a97945af17a76c426de5278bbba8e0ae1cb73994407dd": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:39 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-8h8sv: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6aaaf987021442933f114924aa78aeba028dbb21dfa4fb086778886b3d0c5e81": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:41 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "939d71fee3b0af09a9d98080fd7308beecf5b0d08968ec8e38aad850f09e27e4": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:43 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ada4938b7e17b51594b031b60ae2c4fe0e47edb6c6e4ff39b3c3e0dbeba70139": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:45 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "127493e21a93fc7d1a897951b67d6babdaeb38cb703777678c878e1db0aaa0aa": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:45 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-npbz4: {kubelet i-0ebbe4bd7ec58d213} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:45 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-npbz4: {kubelet i-0ebbe4bd7ec58d213} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:45 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-npbz4: {kubelet i-0ebbe4bd7ec58d213} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:46 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "12c09fce994a06ad1f191a8b2bfcc1f088d771442746252cc3794231aa92de8e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:46 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "04a23c3ca30c7779d2ec108733634161ad25e7b968f2cd5a056e80297f5f402a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:46 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "372a08989452591e69bb2f065972687cfaa1592d962d52bd6a87cb92f67dc97d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:46 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x4pht: {kubelet i-0ebbe4bd7ec58d213} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:46 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x4pht: {kubelet i-0ebbe4bd7ec58d213} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:46 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x4pht: {kubelet i-0ebbe4bd7ec58d213} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:46 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xqz46: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3dda46add283f684cf62da75959718aed27ea39163dcc48ce9f0e7ab2912b6eb": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:46 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f67c4382fc79697da84a2da0b8571271120c1381770347ca31a4419307533010": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:47 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a063ee6f332b4e5ad59dc498c2c46e368e74488369a76d0e640f40a60a6bde80": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:47 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-szf5f: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c7be8eb2769419d4b383fe444bfb6ca2ce35816101569eb9c774a0964bad2765": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:47 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-wm7lb: {kubelet i-0ebbe4bd7ec58d213} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:47 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-wm7lb: {kubelet i-0ebbe4bd7ec58d213} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:47 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-wm7lb: {kubelet i-0ebbe4bd7ec58d213} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:53 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-8h8sv: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "190d9310ea101a9cc885db35efcb1efc22e71059a97fda8d406ebf505cfd8323": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:54 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3515e21b5f65ac6347b28141f3268ac9b33373e44693c4d1dddb7c3b4822d638": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:56 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5230a6d94d4d53ed4d01353189df046b5abb07e1aa3cdd014fd07dfcffe00993": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:56 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "29d3ebe057fb77a32a41e0c4a60047b2679e52785d1504ff1ff24783a4181963": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:58 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "07407e593d8dda194422f66aa5edc1e82b5fd4cfd7305f285c4ca8a829b9fbb9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:59 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ae1b998d31e719072d232a719461af2c08cedd02327ffd064cc8e43f25f803e0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:59 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-szf5f: {kubelet i-0df245ff74f46fa48} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:59 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-szf5f: {kubelet i-0df245ff74f46fa48} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:46:59 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-szf5f: {kubelet i-0df245ff74f46fa48} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:00 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "03cbb5061f68fa739455baac66e95ff1d2346f9e891cbcd9b04dc665d75514ef": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:00 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xqz46: {kubelet i-0dbcb4dc9e1f5e62b} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:00 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xqz46: {kubelet i-0dbcb4dc9e1f5e62b} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:00 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xqz46: {kubelet i-0dbcb4dc9e1f5e62b} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:00 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "375feb51363865c3ce9fcdf549571c81b1be43b622719b5f17ea373d92a30f67": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:01 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "eda6bc1c29d3bbfec758ae38afadc567ca95267edaaa2b7df35b8d24f5a373b3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:06 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-8h8sv: {kubelet i-0dbcb4dc9e1f5e62b} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:06 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-8h8sv: {kubelet i-0dbcb4dc9e1f5e62b} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:06 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-8h8sv: {kubelet i-0dbcb4dc9e1f5e62b} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:07 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ec49280a1282a3270c2dccf1b12c6621112437e3764e0b1755029be7356c1803": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:07 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5f3d66cd9dd8acb8081772d871e310206f2450fb6f6be28cf203fdb8b78e3cd5": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:08 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8e67cb9e3ecf304c0a3c47a898e64f2d1a3bfc0102727a2342997d842eb0a6de": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:09 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4dbb97eb248b7986488f78d5d12c4f890fec9c8f68619b22f8515b3f101d8096": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:11 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "933ae3a662f896c080930f872639bbefd86fe49df430888976698c3b0587f246": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:13 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e22887b2c2390ec7b6ea68d72cf6c1a40f263a8b61ace9b5736702d342a1af11": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:14 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "491387857137ded049958a4aa0522569fbb7fbf55f426576392c94f3c31c3574": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:15 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cddb26ae12fbb2839c2f3c978bb69feeb3275878e909f84dd790a83f48c7bd81": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:19 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5b76973faf6ac4a6e42b94ea869fe9673054b901cd09782de095b3a5b4e8cd47": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:21 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8f129d633794de2318181a0bb003755297bda6964ab50c4c40c3426b0cd34096": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:21 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0c506ab225119a025c138b9977e022188d3c5421e4d48c21b416dfab8095fe2f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:21 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "24cd008fd49c79c7ab179614b4aa8e559e4dd438223131fc7845dae2450a1363": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:25 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8fa5047e0b32b9f1ee9a0d630c25cc7e78f30169c1ce6c6d9542e3649c9cf4b5": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:26 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cf4faecf401c86df7af1893af0f010c312bf103736c8441e1d833826ff53d917": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:27 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4c1f9e14d06bbcc4d224df97a4d4499e91f58d76a933d20fcf2969aa0766aa3c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:30 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ee754d89e37741766b5703815326f1905b0216a1763bb5fa1c9ed6eee8b5a8c5": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:32 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "2fa579f75136e32ff5e4fc658d508a97d2ecfdf145917a902c8a50ff7e74ece7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:34 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6b6f557f5f111cbd139f408407d4757e843fb2c1ab147221e41b20cf2d629f75": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:35 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0336d824ccdb1fc1f34a57114e0204ba82448bd9a7bff7f89297b6366ba34404": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:37 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "37715d9a491bfef528a1cb3a0eb85b17f7b76ad7de65a110edd286e4fc428447": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:40 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc: {kubelet i-0df245ff74f46fa48} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:40 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc: {kubelet i-0df245ff74f46fa48} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:40 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc: {kubelet i-0df245ff74f46fa48} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:40 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f6d72c5bbfbdecd0809f845d43994d1d787865fe7590d6cb2f02e513832de888": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:42 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f5519eb8c90f6b6c6ccdd566a4638ddf58b15206398856de6428921a56ae635f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:43 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4c2dccd424a197638023d93853be8f4d72e826b1b0c2493f03aa24b6f08bab30": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:46 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc: {kubelet i-0e8434bc9e1236fd5} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:46 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc: {kubelet i-0e8434bc9e1236fd5} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:46 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc: {kubelet i-0e8434bc9e1236fd5} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:47 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "52784a12329f3c2c3c0ccd0dba43a42eeaf3c85d06ed4fd9d90f4b41eaf0bd2e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:51 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "10ed95bfb6a015d098247c528fece19bd65f90c54bf8a830b5a5e040f41467a1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:51 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f51f3c442691ce65897cf44e6064c3135c99cd9ec02915e3f2fc83467f08be41": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:52 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1ea600d25695ff9eb3f4457f59c58a8108e52538fa6ab09da87755b9bb4b4db6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:57 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9: {kubelet i-0df245ff74f46fa48} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:57 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9: {kubelet i-0df245ff74f46fa48} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:57 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9: {kubelet i-0df245ff74f46fa48} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:58 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h: {kubelet i-0dbcb4dc9e1f5e62b} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:58 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h: {kubelet i-0dbcb4dc9e1f5e62b} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:58 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h: {kubelet i-0dbcb4dc9e1f5e62b} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:47:58 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f8b2240b9b9f7e3978d52a6fd3cf13b9d9fa2dd9d11ecc3a8d57576c96b756c2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:48:05 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "82c263959a539f592c741d5d0dde224a26fbc7b9a6e8b5bb3ac1f00156ff127e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:48:06 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c144f8b528a11f7a0f302a95af13cd0c420e0d6c2ae67611e9ecbf70596c7bc7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:48:06 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "247f722a085fa6ca264f0b355446d25259513dd3a23ebe2c2127a43f86263ce1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:48:11 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c103c47035a75eff9626dbf38e9fdad5740c196f94b683a294d34c72ccd06003": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:48:16 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5997d2e43e4fcf9df0f604fd562c47d2e1335efd3952d451066f0ae874dc31f8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:48:18 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj: {kubelet i-0dbcb4dc9e1f5e62b} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d8e76ec0600b3539caaea613c6f99cb3428ae79a4ddb89a45684cd1ad3e13972": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:48:21 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj: {kubelet i-0e8434bc9e1236fd5} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1322f8ff3069e62168576136c379df43f46ddd5361c799fccb7881bcd2ce0642": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 25 06:54:25.979: INFO: At 2023-01-25 06:48:24 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt: {kubelet i-0dbcb4dc9e1f5e62b} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:48:24 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt: {kubelet i-0dbcb4dc9e1f5e62b} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:48:25 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt: {kubelet i-0dbcb4dc9e1f5e62b} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:48:42 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h: {kubelet i-0dbcb4dc9e1f5e62b} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:48:42 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h: {kubelet i-0dbcb4dc9e1f5e62b} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:48:42 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h: {kubelet i-0dbcb4dc9e1f5e62b} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:49:22 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj: {kubelet i-0dbcb4dc9e1f5e62b} Pulled: Container image "registry.k8s.io/pause:3.9" already present on machine Jan 25 06:54:25.979: INFO: At 2023-01-25 06:49:22 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj: {kubelet i-0dbcb4dc9e1f5e62b} Created: Created container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:25.979: INFO: At 2023-01-25 06:49:22 +0000 UTC - event for cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj: {kubelet i-0dbcb4dc9e1f5e62b} Started: Started container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 Jan 25 06:54:26.013: INFO: POD NODE PHASE GRACE CONDITIONS Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2kf7l i-0e8434bc9e1236fd5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2mwzt i-0df245ff74f46fa48 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-4wfp7 i-0dbcb4dc9e1f5e62b Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-569r5 i-0df245ff74f46fa48 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:19 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:19 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h i-0dbcb4dc9e1f5e62b Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:47:59 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:47:59 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5mlr8 i-0e8434bc9e1236fd5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-7zg9l i-0ebbe4bd7ec58d213 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:16 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:33 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:33 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:16 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-8h8sv i-0dbcb4dc9e1f5e62b Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:47:07 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:47:07 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9 i-0df245ff74f46fa48 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:47:58 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:47:58 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bpq52 i-0ebbe4bd7ec58d213 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-csrp6 i-0ebbe4bd7ec58d213 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:19 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:19 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt i-0dbcb4dc9e1f5e62b Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:48:25 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:48:25 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ft4g2 i-0ebbe4bd7ec58d213 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:34 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:34 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc i-0df245ff74f46fa48 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:47:41 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:47:41 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h i-0dbcb4dc9e1f5e62b Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:48:43 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:48:43 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hd52p i-0df245ff74f46fa48 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hqhjx i-0ebbe4bd7ec58d213 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:32 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:32 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc i-0e8434bc9e1236fd5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:47:47 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:47:47 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jjlq8 i-0dbcb4dc9e1f5e62b Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jspbp i-0e8434bc9e1236fd5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:31 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:31 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-k9wft i-0ebbe4bd7ec58d213 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:19 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:19 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-lw7tl i-0e8434bc9e1236fd5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:31 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:31 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-npbz4 i-0ebbe4bd7ec58d213 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:45 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:45 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-plznd i-0df245ff74f46fa48 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:16 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:31 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:31 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:16 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-psvwb i-0dbcb4dc9e1f5e62b Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj i-0dbcb4dc9e1f5e62b Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:49:22 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:49:22 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qqns6 i-0e8434bc9e1236fd5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qs6zz i-0ebbe4bd7ec58d213 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:30 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:30 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rbghp i-0e8434bc9e1236fd5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rt5t5 i-0dbcb4dc9e1f5e62b Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rzvq9 i-0df245ff74f46fa48 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:19 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:19 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sr5k2 i-0e8434bc9e1236fd5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:16 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:19 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:19 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:16 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sxzm4 i-0df245ff74f46fa48 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-szf5f i-0df245ff74f46fa48 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:47:00 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:47:00 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-tczwz i-0e8434bc9e1236fd5 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:31 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:31 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-wm7lb i-0ebbe4bd7ec58d213 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:48 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:48 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x4pht i-0ebbe4bd7ec58d213 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:47 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:47 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x5hz9 i-0df245ff74f46fa48 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:18 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xqz46 i-0dbcb4dc9e1f5e62b Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:47:01 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:47:01 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj i-0e8434bc9e1236fd5 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC ContainersNotReady containers with unready status: [cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC ContainersNotReady containers with unready status: [cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:46:15 +0000 UTC }] Jan 25 06:54:26.013: INFO: Jan 25 06:54:26.230: INFO: Logging node info for node i-071cd767a01ed5a98 Jan 25 06:54:26.261: INFO: Node Info: &Node{ObjectMeta:{i-071cd767a01ed5a98 e176a57e-e197-4621-9ff9-1551f1909075 49349 0 2023-01-25 06:25:20 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-071cd767a01ed5a98 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ca-central-1a topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.39.108 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-071cd767a01ed5a98"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-25 06:25:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-25 06:25:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:26:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:26:12 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-25 06:26:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-25 06:53:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-071cd767a01ed5a98,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862937600 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758080000 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:26:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.108,},NodeAddress{Type:ExternalIP,Address:3.97.7.38,},NodeAddress{Type:InternalDNS,Address:i-071cd767a01ed5a98.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-071cd767a01ed5a98.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-97-7-38.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dc6c1622927717fff709d069d68b2,SystemUUID:ec2dc6c1-6229-2771-7fff-709d069d68b2,BootID:034bcfe8-9462-416d-9a80-bb9dcb9fd35d,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.1],SizeBytes:135178704,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.1],SizeBytes:124995897,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:63920476,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.1],SizeBytes:57661752,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:44339025,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42841295,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4965790,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 25 06:54:26.262: INFO: Logging kubelet events for node i-071cd767a01ed5a98 Jan 25 06:54:26.296: INFO: Logging pods the kubelet thinks is on node i-071cd767a01ed5a98 Jan 25 06:54:26.335: INFO: dns-controller-69987775c6-7nx57 started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.335: INFO: Container dns-controller ready: true, restart count 0 Jan 25 06:54:26.335: INFO: aws-cloud-controller-manager-6kpfv started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.335: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 25 06:54:26.335: INFO: etcd-manager-events-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (10+1 container statuses recorded) Jan 25 06:54:26.335: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Container etcd-manager ready: true, restart count 0 Jan 25 06:54:26.335: INFO: etcd-manager-main-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (10+1 container statuses recorded) Jan 25 06:54:26.335: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Container etcd-manager ready: true, restart count 0 Jan 25 06:54:26.335: INFO: kube-apiserver-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+2 container statuses recorded) Jan 25 06:54:26.335: INFO: Container healthcheck ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Container kube-apiserver ready: true, restart count 3 Jan 25 06:54:26.335: INFO: kube-controller-manager-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.335: INFO: Container kube-controller-manager ready: true, restart count 4 Jan 25 06:54:26.335: INFO: kube-scheduler-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.335: INFO: Container kube-scheduler ready: true, restart count 0 Jan 25 06:54:26.335: INFO: cilium-nqkjx started at 2023-01-25 06:26:00 +0000 UTC (1+1 container statuses recorded) Jan 25 06:54:26.335: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Container cilium-agent ready: true, restart count 0 Jan 25 06:54:26.335: INFO: kops-controller-qfzp7 started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.335: INFO: Container kops-controller ready: true, restart count 0 Jan 25 06:54:26.335: INFO: ebs-csi-controller-5bd98b456f-cdjzf started at 2023-01-25 06:26:00 +0000 UTC (0+5 container statuses recorded) Jan 25 06:54:26.335: INFO: Container csi-attacher ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Container csi-provisioner ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Container csi-resizer ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Container ebs-plugin ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Container liveness-probe ready: true, restart count 0 Jan 25 06:54:26.335: INFO: ebs-csi-node-7t2hw started at 2023-01-25 06:26:00 +0000 UTC (0+3 container statuses recorded) Jan 25 06:54:26.335: INFO: Container ebs-plugin ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Container liveness-probe ready: true, restart count 0 Jan 25 06:54:26.335: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 25 06:54:26.335: INFO: cilium-operator-5dd44dc49f-jhvqd started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.335: INFO: Container cilium-operator ready: true, restart count 0 Jan 25 06:54:26.490: INFO: Latency metrics for node i-071cd767a01ed5a98 Jan 25 06:54:26.490: INFO: Logging node info for node i-0dbcb4dc9e1f5e62b Jan 25 06:54:26.522: INFO: Node Info: &Node{ObjectMeta:{i-0dbcb4dc9e1f5e62b f1ad0cae-2ed9-41a4-8be5-9884ae32adaa 49600 0 2023-01-25 06:27:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0dbcb4dc9e1f5e62b kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0dbcb4dc9e1f5e62b topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.63.11 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0dbcb4dc9e1f5e62b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:52:16 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:52:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0dbcb4dc9e1f5e62b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:52:16 +0000 UTC,LastTransitionTime:2023-01-25 06:27:38 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.11,},NodeAddress{Type:ExternalIP,Address:3.96.138.144,},NodeAddress{Type:InternalDNS,Address:i-0dbcb4dc9e1f5e62b.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0dbcb4dc9e1f5e62b.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-138-144.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2cc8535b4596aa012ef7857efd2302,SystemUUID:ec2cc853-5b45-96aa-012e-f7857efd2302,BootID:1d85d5b3-ad1c-462b-98e2-f01921a4e990,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-08b8d4339aaccaee6],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-08b8d4339aaccaee6,DevicePath:,},},Config:nil,},} Jan 25 06:54:26.522: INFO: Logging kubelet events for node i-0dbcb4dc9e1f5e62b Jan 25 06:54:26.557: INFO: Logging pods the kubelet thinks is on node i-0dbcb4dc9e1f5e62b Jan 25 06:54:26.595: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.595: INFO: csi-mockplugin-0 started at 2023-01-25 06:50:15 +0000 UTC (0+3 container statuses recorded) Jan 25 06:54:26.595: INFO: Container csi-provisioner ready: false, restart count 0 Jan 25 06:54:26.595: INFO: Container driver-registrar ready: false, restart count 0 Jan 25 06:54:26.595: INFO: Container mock ready: false, restart count 0 Jan 25 06:54:26.595: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-4wfp7 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.595: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.595: INFO: csi-mockplugin-resizer-0 started at 2023-01-25 06:50:16 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container csi-resizer ready: true, restart count 0 Jan 25 06:54:26.595: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-psvwb started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.595: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.595: INFO: hostexec-i-0dbcb4dc9e1f5e62b-97cc8 started at 2023-01-25 06:50:00 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container agnhost-container ready: true, restart count 0 Jan 25 06:54:26.595: INFO: webserver-deployment-7f5969cbc7-pzv6w started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container httpd ready: false, restart count 0 Jan 25 06:54:26.595: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xqz46 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.595: INFO: ebs-csi-node-bn5vq started at 2023-01-25 06:27:26 +0000 UTC (0+3 container statuses recorded) Jan 25 06:54:26.595: INFO: Container ebs-plugin ready: true, restart count 0 Jan 25 06:54:26.595: INFO: Container liveness-probe ready: true, restart count 0 Jan 25 06:54:26.595: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 25 06:54:26.595: INFO: busybox-1b7825ea-2c0f-4fcb-b7f9-c97bd78ff608 started at 2023-01-25 06:50:33 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container busybox ready: false, restart count 0 Jan 25 06:54:26.595: INFO: webserver-deployment-7f5969cbc7-ctzws started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container httpd ready: false, restart count 0 Jan 25 06:54:26.595: INFO: pod-subpath-test-dynamicpv-gwtk started at 2023-01-25 06:51:51 +0000 UTC (2+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Init container init-volume-dynamicpv-gwtk ready: false, restart count 0 Jan 25 06:54:26.595: INFO: Init container test-init-volume-dynamicpv-gwtk ready: false, restart count 0 Jan 25 06:54:26.595: INFO: Container test-container-subpath-dynamicpv-gwtk ready: false, restart count 0 Jan 25 06:54:26.595: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.595: INFO: cilium-wptjp started at 2023-01-25 06:27:26 +0000 UTC (1+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 25 06:54:26.595: INFO: Container cilium-agent ready: true, restart count 0 Jan 25 06:54:26.595: INFO: local-injector started at 2023-01-25 06:50:06 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container local-injector ready: false, restart count 0 Jan 25 06:54:26.595: INFO: pod-d973dfcb-fb6f-420f-9eb8-855583292025 started at 2023-01-25 06:51:32 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container test-container ready: false, restart count 0 Jan 25 06:54:26.595: INFO: webserver-deployment-7f5969cbc7-fqq2n started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container httpd ready: true, restart count 0 Jan 25 06:54:26.595: INFO: csi-mockplugin-attacher-0 started at 2023-01-25 06:50:16 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container csi-attacher ready: false, restart count 0 Jan 25 06:54:26.595: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jjlq8 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.595: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rt5t5 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.595: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-8h8sv started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.595: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.845: INFO: Latency metrics for node i-0dbcb4dc9e1f5e62b Jan 25 06:54:26.845: INFO: Logging node info for node i-0df245ff74f46fa48 Jan 25 06:54:26.877: INFO: Node Info: &Node{ObjectMeta:{i-0df245ff74f46fa48 3893284f-3580-49f6-b7a1-2ddba9ae06ec 49602 0 2023-01-25 06:27:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0df245ff74f46fa48 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0df245ff74f46fa48 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.41.87 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-6879":"i-0df245ff74f46fa48","ebs.csi.aws.com":"i-0df245ff74f46fa48"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kops-controller Update v1 2023-01-25 06:27:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:50:23 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:50:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0df245ff74f46fa48,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:40 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.87,},NodeAddress{Type:ExternalIP,Address:3.96.184.142,},NodeAddress{Type:InternalDNS,Address:i-0df245ff74f46fa48.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0df245ff74f46fa48.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-184-142.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2129fadd2b2b52f98b14d7c0337fff,SystemUUID:ec2129fa-dd2b-2b52-f98b-14d7c0337fff,BootID:6b9e799a-3b2a-44dc-8a4e-1a86f3b5230b,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-020105115fe204770],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-020105115fe204770,DevicePath:,},},Config:nil,},} Jan 25 06:54:26.877: INFO: Logging kubelet events for node i-0df245ff74f46fa48 Jan 25 06:54:26.912: INFO: Logging pods the kubelet thinks is on node i-0df245ff74f46fa48 Jan 25 06:54:26.957: INFO: webserver-deployment-7f5969cbc7-hrp6g started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Container httpd ready: false, restart count 0 Jan 25 06:54:26.957: INFO: cilium-4kl5t started at 2023-01-25 06:27:28 +0000 UTC (1+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 25 06:54:26.957: INFO: Container cilium-agent ready: true, restart count 0 Jan 25 06:54:26.957: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rzvq9 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.957: INFO: pod-b2d5aa59-ddb2-497b-b629-98cd25649a85 started at 2023-01-25 06:50:17 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Container write-pod ready: false, restart count 0 Jan 25 06:54:26.957: INFO: webserver-deployment-7f5969cbc7-r4gfp started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Container httpd ready: true, restart count 0 Jan 25 06:54:26.957: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-szf5f started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.957: INFO: ebs-csi-node-cfszw started at 2023-01-25 06:27:28 +0000 UTC (0+3 container statuses recorded) Jan 25 06:54:26.957: INFO: Container ebs-plugin ready: true, restart count 0 Jan 25 06:54:26.957: INFO: Container liveness-probe ready: true, restart count 0 Jan 25 06:54:26.957: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 25 06:54:26.957: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x5hz9 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.957: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sxzm4 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.957: INFO: pvc-volume-tester-mrm72 started at 2023-01-25 06:52:23 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Container volume-tester ready: false, restart count 0 Jan 25 06:54:26.957: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.957: INFO: pfpod started at 2023-01-25 06:49:34 +0000 UTC (0+2 container statuses recorded) Jan 25 06:54:26.957: INFO: Container portforwardtester ready: false, restart count 0 Jan 25 06:54:26.957: INFO: Container readiness ready: false, restart count 0 Jan 25 06:54:26.957: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-569r5 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.957: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-plznd started at 2023-01-25 06:46:16 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.957: INFO: csi-mockplugin-0 started at 2023-01-25 06:46:38 +0000 UTC (0+3 container statuses recorded) Jan 25 06:54:26.957: INFO: Container csi-provisioner ready: true, restart count 0 Jan 25 06:54:26.957: INFO: Container driver-registrar ready: true, restart count 0 Jan 25 06:54:26.957: INFO: Container mock ready: true, restart count 0 Jan 25 06:54:26.957: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2mwzt started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.957: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:26.957: INFO: webserver-deployment-7f5969cbc7-q9lv4 started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Container httpd ready: false, restart count 0 Jan 25 06:54:26.957: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hd52p started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:26.957: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.202: INFO: Latency metrics for node i-0df245ff74f46fa48 Jan 25 06:54:27.202: INFO: Logging node info for node i-0e8434bc9e1236fd5 Jan 25 06:54:27.234: INFO: Node Info: &Node{ObjectMeta:{i-0e8434bc9e1236fd5 48f670c2-fe1c-46d3-b197-464836dab89d 49604 0 2023-01-25 06:27:23 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0e8434bc9e1236fd5 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0e8434bc9e1236fd5 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.54.30 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-5972":"i-0e8434bc9e1236fd5","ebs.csi.aws.com":"i-0e8434bc9e1236fd5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:50:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-25 06:52:43 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0e8434bc9e1236fd5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:35 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.30,},NodeAddress{Type:ExternalIP,Address:99.79.58.56,},NodeAddress{Type:InternalDNS,Address:i-0e8434bc9e1236fd5.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0e8434bc9e1236fd5.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-99-79-58-56.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2439e74c65fcfb2f8c6ece88db3944,SystemUUID:ec2439e7-4c65-fcfb-2f8c-6ece88db3944,BootID:568b3843-a4d7-46ef-b502-19f3e8310af2,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-5972^782f81a1-9c7c-11ed-97c4-deff35c7067a kubernetes.io/csi/ebs.csi.aws.com^vol-058ab77a5848fba8a kubernetes.io/csi/ebs.csi.aws.com^vol-06d3e55226f87a3e7],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-06d3e55226f87a3e7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-058ab77a5848fba8a,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-5972^782f81a1-9c7c-11ed-97c4-deff35c7067a,DevicePath:,},},Config:nil,},} Jan 25 06:54:27.234: INFO: Logging kubelet events for node i-0e8434bc9e1236fd5 Jan 25 06:54:27.268: INFO: Logging pods the kubelet thinks is on node i-0e8434bc9e1236fd5 Jan 25 06:54:27.307: INFO: csi-mockplugin-attacher-0 started at 2023-01-25 06:49:32 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container csi-attacher ready: false, restart count 0 Jan 25 06:54:27.307: INFO: csi-hostpathplugin-0 started at 2023-01-25 06:49:50 +0000 UTC (0+7 container statuses recorded) Jan 25 06:54:27.307: INFO: Container csi-attacher ready: true, restart count 0 Jan 25 06:54:27.307: INFO: Container csi-provisioner ready: true, restart count 0 Jan 25 06:54:27.307: INFO: Container csi-resizer ready: true, restart count 0 Jan 25 06:54:27.307: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 25 06:54:27.307: INFO: Container hostpath ready: true, restart count 0 Jan 25 06:54:27.307: INFO: Container liveness-probe ready: true, restart count 0 Jan 25 06:54:27.307: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 25 06:54:27.307: INFO: webserver-deployment-7f5969cbc7-t7ck4 started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container httpd ready: false, restart count 0 Jan 25 06:54:27.307: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sr5k2 started at 2023-01-25 06:46:16 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.307: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rbghp started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.307: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.307: INFO: pod-subpath-test-dynamicpv-mf6j started at 2023-01-25 06:49:53 +0000 UTC (0+2 container statuses recorded) Jan 25 06:54:27.307: INFO: Container test-container-subpath-dynamicpv-mf6j ready: false, restart count 0 Jan 25 06:54:27.307: INFO: Container test-container-volume-dynamicpv-mf6j ready: false, restart count 0 Jan 25 06:54:27.307: INFO: cilium-xkk5h started at 2023-01-25 06:27:24 +0000 UTC (1+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 25 06:54:27.307: INFO: Container cilium-agent ready: true, restart count 0 Jan 25 06:54:27.307: INFO: inline-volume-tester-2shhw started at 2023-01-25 06:47:57 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 25 06:54:27.307: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2kf7l started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.307: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: false, restart count 0 Jan 25 06:54:27.307: INFO: pod-subpath-test-preprovisionedpv-vfbk started at 2023-01-25 06:51:06 +0000 UTC (1+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Init container init-volume-preprovisionedpv-vfbk ready: false, restart count 0 Jan 25 06:54:27.307: INFO: Container test-container-subpath-preprovisionedpv-vfbk ready: false, restart count 0 Jan 25 06:54:27.307: INFO: coredns-559769c974-ggjhw started at 2023-01-25 06:27:35 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container coredns ready: true, restart count 0 Jan 25 06:54:27.307: INFO: hostexec-i-0e8434bc9e1236fd5-9ql96 started at 2023-01-25 06:50:49 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container agnhost-container ready: true, restart count 0 Jan 25 06:54:27.307: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5mlr8 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.307: INFO: csi-mockplugin-0 started at 2023-01-25 06:49:32 +0000 UTC (0+3 container statuses recorded) Jan 25 06:54:27.307: INFO: Container csi-provisioner ready: false, restart count 0 Jan 25 06:54:27.307: INFO: Container driver-registrar ready: false, restart count 0 Jan 25 06:54:27.307: INFO: Container mock ready: false, restart count 0 Jan 25 06:54:27.307: INFO: hostexec-i-0e8434bc9e1236fd5-274ww started at 2023-01-25 06:50:47 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container agnhost-container ready: true, restart count 0 Jan 25 06:54:27.307: INFO: pod-9c25960d-410f-429b-b861-ab98bb2eb98b started at 2023-01-25 06:50:50 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container write-pod ready: false, restart count 0 Jan 25 06:54:27.307: INFO: webserver-deployment-7f5969cbc7-wwshs started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container httpd ready: false, restart count 0 Jan 25 06:54:27.307: INFO: ebs-csi-node-4d5tv started at 2023-01-25 06:27:24 +0000 UTC (0+3 container statuses recorded) Jan 25 06:54:27.307: INFO: Container ebs-plugin ready: true, restart count 0 Jan 25 06:54:27.307: INFO: Container liveness-probe ready: true, restart count 0 Jan 25 06:54:27.307: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 25 06:54:27.307: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qqns6 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.307: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jspbp started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.307: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-lw7tl started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.307: INFO: coredns-autoscaler-7cb5c5b969-mndhj started at 2023-01-25 06:27:35 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container autoscaler ready: true, restart count 0 Jan 25 06:54:27.307: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-tczwz started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.307: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.499: INFO: Latency metrics for node i-0e8434bc9e1236fd5 Jan 25 06:54:27.499: INFO: Logging node info for node i-0ebbe4bd7ec58d213 Jan 25 06:54:27.531: INFO: Node Info: &Node{ObjectMeta:{i-0ebbe4bd7ec58d213 f69d5727-b51d-4595-ab9a-fe57b5c9c863 49608 0 2023-01-25 06:27:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0ebbe4bd7ec58d213 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0ebbe4bd7ec58d213 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.215 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0ebbe4bd7ec58d213"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:49:53 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:49:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0ebbe4bd7ec58d213,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051685376 0} {<nil>} 3956724Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946827776 0} {<nil>} 3854324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:49:56 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:49:56 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:49:56 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:49:56 +0000 UTC,LastTransitionTime:2023-01-25 06:27:38 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.215,},NodeAddress{Type:ExternalIP,Address:3.96.126.35,},NodeAddress{Type:InternalDNS,Address:i-0ebbe4bd7ec58d213.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ebbe4bd7ec58d213.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-126-35.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2da1db9b923c887fdbe1dd4557a0d5,SystemUUID:ec2da1db-9b92-3c88-7fdb-e1dd4557a0d5,BootID:10184fa3-2e15-48fe-9718-382b9d68823e,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-03294c638eea79454 kubernetes.io/csi/ebs.csi.aws.com^vol-0d005d0140f2dadfb],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-03294c638eea79454,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d005d0140f2dadfb,DevicePath:,},},Config:nil,},} Jan 25 06:54:27.533: INFO: Logging kubelet events for node i-0ebbe4bd7ec58d213 Jan 25 06:54:27.568: INFO: Logging pods the kubelet thinks is on node i-0ebbe4bd7ec58d213 Jan 25 06:54:27.607: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hqhjx started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.607: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-csrp6 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.607: INFO: hostexec-i-0ebbe4bd7ec58d213-brbmd started at 2023-01-25 06:51:14 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container agnhost-container ready: true, restart count 0 Jan 25 06:54:27.607: INFO: ebs-csi-node-9ngsq started at 2023-01-25 06:27:27 +0000 UTC (0+3 container statuses recorded) Jan 25 06:54:27.607: INFO: Container ebs-plugin ready: true, restart count 0 Jan 25 06:54:27.607: INFO: Container liveness-probe ready: true, restart count 0 Jan 25 06:54:27.607: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 25 06:54:27.607: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bpq52 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.607: INFO: pod-disruption-failure-ignore-0-l6qmf started at 2023-01-25 06:46:49 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container c ready: true, restart count 0 Jan 25 06:54:27.607: INFO: pod-disruption-failure-ignore-1-f77bm started at 2023-01-25 06:46:49 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container c ready: false, restart count 0 Jan 25 06:54:27.607: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-k9wft started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.607: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qs6zz started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.607: INFO: cilium-pkfh6 started at 2023-01-25 06:27:27 +0000 UTC (1+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 25 06:54:27.607: INFO: Container cilium-agent ready: true, restart count 0 Jan 25 06:54:27.607: INFO: coredns-559769c974-v87jf started at 2023-01-25 06:27:57 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container coredns ready: true, restart count 0 Jan 25 06:54:27.607: INFO: inline-volume-tester-bns44 started at 2023-01-25 06:49:50 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 25 06:54:27.607: INFO: webserver-deployment-7f5969cbc7-wch65 started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container httpd ready: false, restart count 0 Jan 25 06:54:27.607: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-npbz4 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.607: INFO: pod-disruption-failure-ignore-1-vmrs8 started at 2023-01-25 06:47:22 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container c ready: false, restart count 0 Jan 25 06:54:27.607: INFO: csi-hostpathplugin-0 started at 2023-01-25 06:49:47 +0000 UTC (0+7 container statuses recorded) Jan 25 06:54:27.607: INFO: Container csi-attacher ready: false, restart count 0 Jan 25 06:54:27.607: INFO: Container csi-provisioner ready: false, restart count 0 Jan 25 06:54:27.607: INFO: Container csi-resizer ready: false, restart count 0 Jan 25 06:54:27.607: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 25 06:54:27.607: INFO: Container hostpath ready: false, restart count 0 Jan 25 06:54:27.607: INFO: Container liveness-probe ready: false, restart count 0 Jan 25 06:54:27.607: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 25 06:54:27.607: INFO: pod-7dc5023a-bc51-44e0-a5b1-2ac8cbb4d9a7 started at 2023-01-25 06:51:20 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container write-pod ready: false, restart count 0 Jan 25 06:54:27.607: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-wm7lb started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.607: INFO: pod-77e2c30c-da83-47c3-846b-f0c680a63e2d started at 2023-01-25 06:49:59 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container write-pod ready: false, restart count 0 Jan 25 06:54:27.607: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-7zg9l started at 2023-01-25 06:46:16 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.607: INFO: webserver-deployment-7f5969cbc7-wg5sj started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container httpd ready: false, restart count 0 Jan 25 06:54:27.607: INFO: external-injector started at 2023-01-25 06:49:33 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container external-injector ready: false, restart count 0 Jan 25 06:54:27.607: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x4pht started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.607: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ft4g2 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0 Jan 25 06:54:27.607: INFO: hostexec-i-0ebbe4bd7ec58d213-9jht8 started at 2023-01-25 06:49:56 +0000 UTC (0+1 container statuses recorded) Jan 25 06:54:27.607: INFO: Container agnhost-container ready: true, restart count 0 Jan 25 06:54:27.877: INFO: Latency metrics for node i-0ebbe4bd7ec58d213 [DeferCleanup (Each)] [sig-node] kubelet tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "kubelet-9040" for this suite. �[38;5;243m01/25/23 06:54:27.877�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-storage\]\sCSI\sVolumes\s\[Driver\:\scsi\-hostpath\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\]\sprovisioning\sshould\sprovision\sstorage\swith\spvc\sdata\ssource$'
test/e2e/framework/volume/fixtures.go:571
k8s.io/kubernetes/test/e2e/framework/volume.InjectContent(0xc000e44f00, {{0xc00280dde8, 0x11}, {0x75c7bd6, 0x8}, {0x0, 0x0}, {0x0, 0x0, 0x0}, ...}, ...)
test/e2e/framework/volume/fixtures.go:571 +0x279
k8s.io/kubernetes/test/e2e/storage/testsuites.preparePVCDataSourceForProvisioning({0x7feecbed9b98, 0xc0018224c0}, 0x0?, {{0xc00280dde8, 0x11}, {0x75c7bd6, 0x8}, {0x0, 0x0}, {0x0, ...}, ...}, ...)
test/e2e/storage/testsuites/provisioning.go:1131 +0x305
k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func5({0x7feecbed9b98, 0xc0018224c0})
test/e2e/storage/testsuites/provisioning.go:430 +0x37b
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] provisioning
test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] provisioning
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/25/23 06:49:45.278�[0m
Jan 25 06:49:45.278: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename provisioning �[38;5;243m01/25/23 06:49:45.279�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/25/23 06:49:45.374�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/25/23 06:49:45.435�[0m
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] provisioning
test/e2e/framework/metrics/init/init.go:31
[It] should provision storage with pvc data source
test/e2e/storage/testsuites/provisioning.go:415
�[1mSTEP:�[0m Building a driver namespace object, basename provisioning-1869 �[38;5;243m01/25/23 06:49:45.496�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/25/23 06:49:45.59�[0m
�[1mSTEP:�[0m deploying csi-hostpath driver �[38;5;243m01/25/23 06:49:45.65�[0m
Jan 25 06:49:45.781: INFO: creating *v1.ServiceAccount: provisioning-1869-9392/csi-attacher
Jan 25 06:49:45.813: INFO: creating *v1.ClusterRole: external-attacher-runner-provisioning-1869
Jan 25 06:49:45.813: INFO: Define cluster role external-attacher-runner-provisioning-1869
Jan 25 06:49:45.845: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-provisioning-1869
Jan 25 06:49:45.878: INFO: creating *v1.Role: provisioning-1869-9392/external-attacher-cfg-provisioning-1869
Jan 25 06:49:45.909: INFO: creating *v1.RoleBinding: provisioning-1869-9392/csi-attacher-role-cfg
Jan 25 06:49:45.941: INFO: creating *v1.ServiceAccount: provisioning-1869-9392/csi-provisioner
Jan 25 06:49:45.973: INFO: creating *v1.ClusterRole: external-provisioner-runner-provisioning-1869
Jan 25 06:49:45.973: INFO: Define cluster role external-provisioner-runner-provisioning-1869
Jan 25 06:49:46.005: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-provisioning-1869
Jan 25 06:49:46.038: INFO: creating *v1.Role: provisioning-1869-9392/external-provisioner-cfg-provisioning-1869
Jan 25 06:49:46.070: INFO: creating *v1.RoleBinding: provisioning-1869-9392/csi-provisioner-role-cfg
Jan 25 06:49:46.102: INFO: creating *v1.ServiceAccount: provisioning-1869-9392/csi-snapshotter
Jan 25 06:49:46.133: INFO: creating *v1.ClusterRole: external-snapshotter-runner-provisioning-1869
Jan 25 06:49:46.133: INFO: Define cluster role external-snapshotter-runner-provisioning-1869
Jan 25 06:49:46.166: INFO: creating *v1.ClusterRoleBinding: csi-snapshotter-role-provisioning-1869
Jan 25 06:49:46.198: INFO: creating *v1.Role: provisioning-1869-9392/external-snapshotter-leaderelection-provisioning-1869
Jan 25 06:49:46.230: INFO: creating *v1.RoleBinding: provisioning-1869-9392/external-snapshotter-leaderelection
Jan 25 06:49:46.262: INFO: creating *v1.ServiceAccount: provisioning-1869-9392/csi-external-health-monitor-controller
Jan 25 06:49:46.294: INFO: creating *v1.ClusterRole: external-health-monitor-controller-runner-provisioning-1869
Jan 25 06:49:46.294: INFO: Define cluster role external-health-monitor-controller-runner-provisioning-1869
Jan 25 06:49:46.325: INFO: creating *v1.ClusterRoleBinding: csi-external-health-monitor-controller-role-provisioning-1869
Jan 25 06:49:46.357: INFO: creating *v1.Role: provisioning-1869-9392/external-health-monitor-controller-cfg-provisioning-1869
Jan 25 06:49:46.389: INFO: creating *v1.RoleBinding: provisioning-1869-9392/csi-external-health-monitor-controller-role-cfg
Jan 25 06:49:46.422: INFO: creating *v1.ServiceAccount: provisioning-1869-9392/csi-resizer
Jan 25 06:49:46.454: INFO: creating *v1.ClusterRole: external-resizer-runner-provisioning-1869
Jan 25 06:49:46.454: INFO: Define cluster role external-resizer-runner-provisioning-1869
Jan 25 06:49:46.485: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-provisioning-1869
Jan 25 06:49:46.518: INFO: creating *v1.Role: provisioning-1869-9392/external-resizer-cfg-provisioning-1869
Jan 25 06:49:46.550: INFO: creating *v1.RoleBinding: provisioning-1869-9392/csi-resizer-role-cfg
Jan 25 06:49:46.582: INFO: creating *v1.CSIDriver: csi-hostpath-provisioning-1869
Jan 25 06:49:46.615: INFO: creating *v1.ServiceAccount: provisioning-1869-9392/csi-hostpathplugin-sa
Jan 25 06:49:46.647: INFO: creating *v1.ClusterRoleBinding: csi-hostpathplugin-attacher-cluster-role-provisioning-1869
Jan 25 06:49:46.679: INFO: creating *v1.ClusterRoleBinding: csi-hostpathplugin-health-monitor-controller-cluster-role-provisioning-1869
Jan 25 06:49:46.714: INFO: creating *v1.ClusterRoleBinding: csi-hostpathplugin-provisioner-cluster-role-provisioning-1869
Jan 25 06:49:46.746: INFO: creating *v1.ClusterRoleBinding: csi-hostpathplugin-resizer-cluster-role-provisioning-1869
Jan 25 06:49:46.777: INFO: creating *v1.ClusterRoleBinding: csi-hostpathplugin-snapshotter-cluster-role-provisioning-1869
Jan 25 06:49:46.809: INFO: creating *v1.RoleBinding: provisioning-1869-9392/csi-hostpathplugin-attacher-role
Jan 25 06:49:46.841: INFO: creating *v1.RoleBinding: provisioning-1869-9392/csi-hostpathplugin-health-monitor-controller-role
Jan 25 06:49:46.873: INFO: creating *v1.RoleBinding: provisioning-1869-9392/csi-hostpathplugin-provisioner-role
Jan 25 06:49:46.906: INFO: creating *v1.RoleBinding: provisioning-1869-9392/csi-hostpathplugin-resizer-role
Jan 25 06:49:46.939: INFO: creating *v1.RoleBinding: provisioning-1869-9392/csi-hostpathplugin-snapshotter-role
Jan 25 06:49:46.971: INFO: creating *v1.StatefulSet: provisioning-1869-9392/csi-hostpathplugin
Jan 25 06:49:47.005: INFO: creating *v1.ClusterRoleBinding: psp-csi-hostpath-role-provisioning-1869
Jan 25 06:49:47.042: INFO: In creating storage class object and pvc objects for driver - sc: &StorageClass{ObjectMeta:{provisioning-1869zmt5p 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Provisioner:csi-hostpath-provisioning-1869,Parameters:map[string]string{},ReclaimPolicy:nil,MountOptions:[],AllowVolumeExpansion:nil,VolumeBindingMode:*Immediate,AllowedTopologies:[]TopologySelectorTerm{},}, pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-1869 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1048576 0} {<nil>} 1Mi BinarySI},},Claims:[]ResourceClaim{},},VolumeName:,Selector:nil,StorageClassName:*provisioning-1869zmt5p,VolumeMode:*Block,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},}, src-pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-1869 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1048576 0} {<nil>} 1Mi BinarySI},},Claims:[]ResourceClaim{},},VolumeName:,Selector:nil,StorageClassName:*provisioning-1869zmt5p,VolumeMode:*Block,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},}
�[1mSTEP:�[0m Creating a StorageClass �[38;5;243m01/25/23 06:49:47.073�[0m
�[1mSTEP:�[0m [Initialize dataSource]creating a source PVC �[38;5;243m01/25/23 06:49:47.135�[0m
�[1mSTEP:�[0m starting hostpath-injector �[38;5;243m01/25/23 06:49:47.168�[0m
Jan 25 06:49:47.200: INFO: Waiting up to 5m0s for pod "hostpath-injector" in namespace "provisioning-1869" to be "running"
Jan 25 06:49:47.231: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 30.988115ms
Jan 25 06:49:49.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2.062775365s
Jan 25 06:49:51.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4.063064068s
Jan 25 06:49:53.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 6.063190843s
Jan 25 06:49:55.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 8.062708511s
Jan 25 06:49:57.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 10.063159456s
Jan 25 06:49:59.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 12.063049283s
Jan 25 06:50:01.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 14.062883842s
Jan 25 06:50:03.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 16.062951438s
Jan 25 06:50:05.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 18.063190002s
Jan 25 06:50:07.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 20.062978475s
Jan 25 06:50:09.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 22.063085563s
Jan 25 06:50:11.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 24.063072733s
Jan 25 06:50:13.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 26.062935819s
Jan 25 06:50:15.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 28.062827826s
Jan 25 06:50:17.266: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 30.065703044s
Jan 25 06:50:19.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 32.063284368s
Jan 25 06:50:21.267: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 34.066229687s
Jan 25 06:50:23.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 36.063382586s
Jan 25 06:50:25.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 38.06232525s
Jan 25 06:50:27.268: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 40.067153585s
Jan 25 06:50:29.266: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 42.065489535s
Jan 25 06:50:31.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 44.063174367s
Jan 25 06:50:33.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 46.062808701s
Jan 25 06:50:35.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 48.062567925s
Jan 25 06:50:37.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 50.063187074s
Jan 25 06:50:39.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 52.063491927s
Jan 25 06:50:41.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 54.062506685s
Jan 25 06:50:43.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 56.063549641s
Jan 25 06:50:45.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 58.063519271s
Jan 25 06:50:47.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.063290026s
Jan 25 06:50:49.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.063147735s
Jan 25 06:50:51.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.062618646s
Jan 25 06:50:53.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.063104119s
Jan 25 06:50:55.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.062267035s
Jan 25 06:50:57.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.063170824s
Jan 25 06:50:59.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.063069783s
Jan 25 06:51:01.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.063231175s
Jan 25 06:51:03.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.062445985s
Jan 25 06:51:05.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.06288201s
Jan 25 06:51:07.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.062955995s
Jan 25 06:51:09.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.06330276s
Jan 25 06:51:11.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.063228304s
Jan 25 06:51:13.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.062541124s
Jan 25 06:51:15.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.063772633s
Jan 25 06:51:17.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.062579295s
Jan 25 06:51:19.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.063416692s
Jan 25 06:51:21.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.062845406s
Jan 25 06:51:23.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.063482671s
Jan 25 06:51:25.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.062885544s
Jan 25 06:51:27.268: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.067168534s
Jan 25 06:51:29.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.06318538s
Jan 25 06:51:31.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.063389046s
Jan 25 06:51:33.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.06250871s
Jan 25 06:51:35.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.062478816s
Jan 25 06:51:37.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.062430186s
Jan 25 06:51:39.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.062544746s
Jan 25 06:51:41.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.062916822s
Jan 25 06:51:43.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.062398246s
Jan 25 06:51:45.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.062998735s
Jan 25 06:51:47.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.063996998s
Jan 25 06:51:49.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.063877358s
Jan 25 06:51:51.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.063223074s
Jan 25 06:51:53.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.063554413s
Jan 25 06:51:55.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.062891005s
Jan 25 06:51:57.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.063554013s
Jan 25 06:51:59.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.063389414s
Jan 25 06:52:01.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.063779252s
Jan 25 06:52:03.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.063904123s
Jan 25 06:52:05.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.063317413s
Jan 25 06:52:07.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.063320756s
Jan 25 06:52:09.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.063266916s
Jan 25 06:52:11.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.063272199s
Jan 25 06:52:13.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.063669041s
Jan 25 06:52:15.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.062597592s
Jan 25 06:52:17.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.062476526s
Jan 25 06:52:19.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.063228412s
Jan 25 06:52:21.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.063462252s
Jan 25 06:52:23.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.062887481s
Jan 25 06:52:25.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.06269683s
Jan 25 06:52:27.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.06309404s
Jan 25 06:52:29.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.063373954s
Jan 25 06:52:31.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.06364178s
Jan 25 06:52:33.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.062688625s
Jan 25 06:52:35.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.063521774s
Jan 25 06:52:37.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.062871744s
Jan 25 06:52:39.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.062684613s
Jan 25 06:52:41.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.062843507s
Jan 25 06:52:43.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.062487764s
Jan 25 06:52:45.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.062897222s
Jan 25 06:52:47.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.0626213s
Jan 25 06:52:49.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.06246798s
Jan 25 06:52:51.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.062532403s
Jan 25 06:52:53.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.063421244s
Jan 25 06:52:55.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.063095393s
Jan 25 06:52:57.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.063492136s
Jan 25 06:52:59.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.063182978s
Jan 25 06:53:01.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.063423504s
Jan 25 06:53:03.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.063376226s
Jan 25 06:53:05.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.062909494s
Jan 25 06:53:07.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.062997051s
Jan 25 06:53:09.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.062912168s
Jan 25 06:53:11.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.063215558s
Jan 25 06:53:13.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.062160259s
Jan 25 06:53:15.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.063070618s
Jan 25 06:53:17.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.063725735s
Jan 25 06:53:19.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.062902265s
Jan 25 06:53:21.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.062970606s
Jan 25 06:53:23.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.062549743s
Jan 25 06:53:25.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.06290536s
Jan 25 06:53:27.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.063206052s
Jan 25 06:53:29.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.063429901s
Jan 25 06:53:31.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.062564339s
Jan 25 06:53:33.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.062510044s
Jan 25 06:53:35.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.063021331s
Jan 25 06:53:37.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.062375127s
Jan 25 06:53:39.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.062290524s
Jan 25 06:53:41.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.062964806s
Jan 25 06:53:43.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.062400448s
Jan 25 06:53:45.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.062942485s
Jan 25 06:53:47.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.062347277s
Jan 25 06:53:49.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.063334972s
Jan 25 06:53:51.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.063280373s
Jan 25 06:53:53.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.062594362s
Jan 25 06:53:55.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.063161696s
Jan 25 06:53:57.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.063018088s
Jan 25 06:53:59.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.063249398s
Jan 25 06:54:01.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.063635172s
Jan 25 06:54:03.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.062842047s
Jan 25 06:54:05.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.062674325s
Jan 25 06:54:07.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.063201106s
Jan 25 06:54:09.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.063408937s
Jan 25 06:54:11.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.063291165s
Jan 25 06:54:13.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.062607352s
Jan 25 06:54:15.265: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.064389459s
Jan 25 06:54:17.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.063373461s
Jan 25 06:54:19.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.063270216s
Jan 25 06:54:21.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.06295094s
Jan 25 06:54:23.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.063108244s
Jan 25 06:54:25.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.062731006s
Jan 25 06:54:27.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.063076344s
Jan 25 06:54:29.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.062983s
Jan 25 06:54:31.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.062685566s
Jan 25 06:54:33.274: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.073578159s
Jan 25 06:54:35.264: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.063496307s
Jan 25 06:54:37.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.06248183s
Jan 25 06:54:39.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.062517303s
Jan 25 06:54:41.265: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.064944826s
Jan 25 06:54:43.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.062469415s
Jan 25 06:54:45.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.062531594s
Jan 25 06:54:47.263: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.062896633s
Jan 25 06:54:47.294: INFO: Pod "hostpath-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.093932299s
�[1mSTEP:�[0m Deleting pod hostpath-injector in namespace provisioning-1869 �[38;5;243m01/25/23 06:54:47.294�[0m
Jan 25 06:54:47.328: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:54:47.359: INFO: Pod hostpath-injector still exists
Jan 25 06:54:49.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:54:49.393: INFO: Pod hostpath-injector still exists
Jan 25 06:54:51.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:54:51.391: INFO: Pod hostpath-injector still exists
Jan 25 06:54:53.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:54:53.391: INFO: Pod hostpath-injector still exists
Jan 25 06:54:55.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:54:55.395: INFO: Pod hostpath-injector still exists
Jan 25 06:54:57.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:54:57.391: INFO: Pod hostpath-injector still exists
Jan 25 06:54:59.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:54:59.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:01.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:01.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:03.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:03.392: INFO: Pod hostpath-injector still exists
Jan 25 06:55:05.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:05.395: INFO: Pod hostpath-injector still exists
Jan 25 06:55:07.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:07.392: INFO: Pod hostpath-injector still exists
Jan 25 06:55:09.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:09.394: INFO: Pod hostpath-injector still exists
Jan 25 06:55:11.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:11.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:13.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:13.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:15.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:15.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:17.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:17.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:19.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:19.392: INFO: Pod hostpath-injector still exists
Jan 25 06:55:21.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:21.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:23.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:23.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:25.361: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:25.392: INFO: Pod hostpath-injector still exists
Jan 25 06:55:27.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:27.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:29.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:29.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:31.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:31.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:33.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:33.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:35.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:35.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:37.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:37.393: INFO: Pod hostpath-injector still exists
Jan 25 06:55:39.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:39.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:41.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:41.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:43.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:43.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:45.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:45.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:47.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:47.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:49.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:49.392: INFO: Pod hostpath-injector still exists
Jan 25 06:55:51.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:51.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:53.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:53.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:55.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:55.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:57.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:57.391: INFO: Pod hostpath-injector still exists
Jan 25 06:55:59.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:55:59.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:01.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:01.391: INFO: Pod hostpath-injector still exists
Jan 25 06:56:03.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:03.391: INFO: Pod hostpath-injector still exists
Jan 25 06:56:05.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:05.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:07.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:07.391: INFO: Pod hostpath-injector still exists
Jan 25 06:56:09.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:09.391: INFO: Pod hostpath-injector still exists
Jan 25 06:56:11.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:11.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:13.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:13.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:15.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:15.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:17.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:17.391: INFO: Pod hostpath-injector still exists
Jan 25 06:56:19.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:19.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:21.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:21.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:23.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:23.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:25.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:25.391: INFO: Pod hostpath-injector still exists
Jan 25 06:56:27.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:27.391: INFO: Pod hostpath-injector still exists
Jan 25 06:56:29.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:29.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:31.361: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:31.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:33.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:33.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:35.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:35.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:37.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:37.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:39.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:39.391: INFO: Pod hostpath-injector still exists
Jan 25 06:56:41.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:41.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:43.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:43.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:45.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:45.391: INFO: Pod hostpath-injector still exists
Jan 25 06:56:47.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:47.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:49.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:49.392: INFO: Pod hostpath-injector still exists
Jan 25 06:56:51.360: INFO: Waiting for pod hostpath-injector to disappear
Jan 25 06:56:51.391: INFO: Pod hostpath-injector no longer exists
Jan 25 06:56:51.391: FAIL: Failed to create injector pod: timed out while waiting for pod provisioning-1869/hostpath-injector to be running
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/volume.InjectContent(0xc000e44f00, {{0xc00280dde8, 0x11}, {0x75c7bd6, 0x8}, {0x0, 0x0}, {0x0, 0x0, 0x0}, ...}, ...)
test/e2e/framework/volume/fixtures.go:571 +0x279
k8s.io/kubernetes/test/e2e/storage/testsuites.preparePVCDataSourceForProvisioning({0x7feecbed9b98, 0xc0018224c0}, 0x0?, {{0xc00280dde8, 0x11}, {0x75c7bd6, 0x8}, {0x0, 0x0}, {0x0, ...}, ...}, ...)
test/e2e/storage/testsuites/provisioning.go:1131 +0x305
k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func5({0x7feecbed9b98, 0xc0018224c0})
test/e2e/storage/testsuites/provisioning.go:430 +0x37b
[AfterEach] [Testpattern: Dynamic PV (block volmode)] provisioning
test/e2e/framework/node/init/init.go:32
Jan 25 06:56:51.391: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [Testpattern: Dynamic PV (block volmode)] provisioning
test/e2e/storage/testsuites/provisioning.go:561
Jan 25 06:56:51.424: INFO: deleting storage class provisioning-1869zmt5p
[DeferCleanup (Each)] [Testpattern: Dynamic PV (block volmode)] provisioning
test/e2e/storage/testsuites/provisioning.go:145
[DeferCleanup (Each)] [Testpattern: Dynamic PV (block volmode)] provisioning
test/e2e/storage/drivers/csi.go:289
�[1mSTEP:�[0m deleting the test namespace: provisioning-1869 �[38;5;243m01/25/23 06:56:51.458�[0m
�[1mSTEP:�[0m Collecting events from namespace "provisioning-1869". �[38;5;243m01/25/23 06:56:51.458�[0m
�[1mSTEP:�[0m Found 7 events. �[38;5;243m01/25/23 06:56:51.49�[0m
Jan 25 06:56:51.490: INFO: At 2023-01-25 06:49:47 +0000 UTC - event for hostpath-injector: {default-scheduler } FailedScheduling: 0/5 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/5 nodes are available: 5 No preemption victims found for incoming pod..
Jan 25 06:56:51.490: INFO: At 2023-01-25 06:49:47 +0000 UTC - event for pvc-htg8t: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "csi-hostpath-provisioning-1869" or manually created by system administrator
Jan 25 06:56:51.490: INFO: At 2023-01-25 06:54:44 +0000 UTC - event for pvc-htg8t: {csi-hostpath-provisioning-1869_csi-hostpathplugin-0_9f2080b7-57a9-4e67-a6de-ece10d29384a } Provisioning: External provisioner is provisioning volume for claim "provisioning-1869/pvc-htg8t"
Jan 25 06:56:51.490: INFO: At 2023-01-25 06:54:44 +0000 UTC - event for pvc-htg8t: {csi-hostpath-provisioning-1869_csi-hostpathplugin-0_9f2080b7-57a9-4e67-a6de-ece10d29384a } ProvisioningSucceeded: Successfully provisioned volume pvc-c9646542-1210-444f-9790-28a0662f3529
Jan 25 06:56:51.490: INFO: At 2023-01-25 06:54:46 +0000 UTC - event for hostpath-injector: {default-scheduler } Scheduled: Successfully assigned provisioning-1869/hostpath-injector to i-0ebbe4bd7ec58d213
Jan 25 06:56:51.490: INFO: At 2023-01-25 06:54:46 +0000 UTC - event for hostpath-injector: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-c9646542-1210-444f-9790-28a0662f3529"
Jan 25 06:56:51.490: INFO: At 2023-01-25 06:56:49 +0000 UTC - event for hostpath-injector: {kubelet i-0ebbe4bd7ec58d213} FailedMount: Unable to attach or mount volumes: unmounted volumes=[hostpath-volume-0], unattached volumes=[kube-api-access-xzg9n hostpath-volume-0]: timed out waiting for the condition
Jan 25 06:56:51.521: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 25 06:56:51.521: INFO:
Jan 25 06:56:51.555: INFO:
Logging node info for node i-071cd767a01ed5a98
Jan 25 06:56:51.586: INFO: Node Info: &Node{ObjectMeta:{i-071cd767a01ed5a98 e176a57e-e197-4621-9ff9-1551f1909075 49349 0 2023-01-25 06:25:20 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-071cd767a01ed5a98 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ca-central-1a topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.39.108 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-071cd767a01ed5a98"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-25 06:25:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-25 06:25:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:26:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:26:12 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-25 06:26:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-25 06:53:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-071cd767a01ed5a98,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862937600 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758080000 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:26:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.108,},NodeAddress{Type:ExternalIP,Address:3.97.7.38,},NodeAddress{Type:InternalDNS,Address:i-071cd767a01ed5a98.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-071cd767a01ed5a98.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-97-7-38.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dc6c1622927717fff709d069d68b2,SystemUUID:ec2dc6c1-6229-2771-7fff-709d069d68b2,BootID:034bcfe8-9462-416d-9a80-bb9dcb9fd35d,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.1],SizeBytes:135178704,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.1],SizeBytes:124995897,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:63920476,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.1],SizeBytes:57661752,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:44339025,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42841295,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4965790,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 25 06:56:51.586: INFO:
Logging kubelet events for node i-071cd767a01ed5a98
Jan 25 06:56:51.620: INFO:
Logging pods the kubelet thinks is on node i-071cd767a01ed5a98
Jan 25 06:56:51.671: INFO: cilium-nqkjx started at 2023-01-25 06:26:00 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:56:51.671: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:56:51.671: INFO: dns-controller-69987775c6-7nx57 started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:56:51.671: INFO: Container dns-controller ready: true, restart count 0
Jan 25 06:56:51.671: INFO: aws-cloud-controller-manager-6kpfv started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:56:51.671: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 25 06:56:51.671: INFO: etcd-manager-events-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (10+1 container statuses recorded)
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Container etcd-manager ready: true, restart count 0
Jan 25 06:56:51.671: INFO: etcd-manager-main-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (10+1 container statuses recorded)
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Container etcd-manager ready: true, restart count 0
Jan 25 06:56:51.671: INFO: kube-apiserver-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+2 container statuses recorded)
Jan 25 06:56:51.671: INFO: Container healthcheck ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Container kube-apiserver ready: true, restart count 3
Jan 25 06:56:51.671: INFO: kube-controller-manager-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:56:51.671: INFO: Container kube-controller-manager ready: true, restart count 4
Jan 25 06:56:51.671: INFO: kube-scheduler-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:56:51.671: INFO: Container kube-scheduler ready: true, restart count 0
Jan 25 06:56:51.671: INFO: kops-controller-qfzp7 started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:56:51.671: INFO: Container kops-controller ready: true, restart count 0
Jan 25 06:56:51.671: INFO: ebs-csi-controller-5bd98b456f-cdjzf started at 2023-01-25 06:26:00 +0000 UTC (0+5 container statuses recorded)
Jan 25 06:56:51.671: INFO: Container csi-attacher ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Container csi-resizer ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:56:51.671: INFO: ebs-csi-node-7t2hw started at 2023-01-25 06:26:00 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:56:51.671: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:56:51.671: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:56:51.671: INFO: cilium-operator-5dd44dc49f-jhvqd started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:56:51.671: INFO: Container cilium-operator ready: true, restart count 0
Jan 25 06:56:51.835: INFO:
Latency metrics for node i-071cd767a01ed5a98
Jan 25 06:56:51.835: INFO:
Logging node info for node i-0dbcb4dc9e1f5e62b
Jan 25 06:56:51.867: INFO: Node Info: &Node{ObjectMeta:{i-0dbcb4dc9e1f5e62b f1ad0cae-2ed9-41a4-8be5-9884ae32adaa 51044 0 2023-01-25 06:27:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0dbcb4dc9e1f5e62b kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0dbcb4dc9e1f5e62b topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.63.11 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-9383":"i-0dbcb4dc9e1f5e62b","ebs.csi.aws.com":"i-0dbcb4dc9e1f5e62b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:55:59 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:56:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0dbcb4dc9e1f5e62b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:56:00 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:56:00 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:56:00 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:56:00 +0000 UTC,LastTransitionTime:2023-01-25 06:27:38 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.11,},NodeAddress{Type:ExternalIP,Address:3.96.138.144,},NodeAddress{Type:InternalDNS,Address:i-0dbcb4dc9e1f5e62b.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0dbcb4dc9e1f5e62b.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-138-144.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2cc8535b4596aa012ef7857efd2302,SystemUUID:ec2cc853-5b45-96aa-012e-f7857efd2302,BootID:1d85d5b3-ad1c-462b-98e2-f01921a4e990,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-9383^5258fd8d-9c7d-11ed-aea5-a678623d062f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-9383^5258fd8d-9c7d-11ed-aea5-a678623d062f,DevicePath:,},},Config:nil,},}
Jan 25 06:56:51.867: INFO:
Logging kubelet events for node i-0dbcb4dc9e1f5e62b
Jan 25 06:56:51.901: INFO:
Logging pods the kubelet thinks is on node i-0dbcb4dc9e1f5e62b
Jan 25 06:56:51.944: INFO: ebs-csi-node-bn5vq started at 2023-01-25 06:27:26 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:56:51.944: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:56:51.944: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:56:51.944: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:56:51.944: INFO: cilium-wptjp started at 2023-01-25 06:27:26 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:56:51.944: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:56:51.944: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:56:51.944: INFO: csi-mockplugin-attacher-0 started at 2023-01-25 06:50:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:56:51.944: INFO: Container csi-attacher ready: true, restart count 0
Jan 25 06:56:51.944: INFO: csi-mockplugin-0 started at 2023-01-25 06:50:15 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:56:51.944: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:56:51.944: INFO: Container driver-registrar ready: true, restart count 0
Jan 25 06:56:51.944: INFO: Container mock ready: true, restart count 0
Jan 25 06:56:51.944: INFO: csi-mockplugin-resizer-0 started at 2023-01-25 06:50:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:56:51.944: INFO: Container csi-resizer ready: true, restart count 0
Jan 25 06:56:51.944: INFO: pvc-volume-tester-rx8qh started at 2023-01-25 06:55:58 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:56:51.944: INFO: Container volume-tester ready: true, restart count 0
Jan 25 06:56:52.164: INFO:
Latency metrics for node i-0dbcb4dc9e1f5e62b
Jan 25 06:56:52.164: INFO:
Logging node info for node i-0df245ff74f46fa48
Jan 25 06:56:52.196: INFO: Node Info: &Node{ObjectMeta:{i-0df245ff74f46fa48 3893284f-3580-49f6-b7a1-2ddba9ae06ec 50214 0 2023-01-25 06:27:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0df245ff74f46fa48 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0df245ff74f46fa48 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.41.87 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0df245ff74f46fa48"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kops-controller Update v1 2023-01-25 06:27:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-25 06:54:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0df245ff74f46fa48,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:54:52 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:54:52 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:54:52 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:54:52 +0000 UTC,LastTransitionTime:2023-01-25 06:27:40 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.87,},NodeAddress{Type:ExternalIP,Address:3.96.184.142,},NodeAddress{Type:InternalDNS,Address:i-0df245ff74f46fa48.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0df245ff74f46fa48.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-184-142.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2129fadd2b2b52f98b14d7c0337fff,SystemUUID:ec2129fa-dd2b-2b52-f98b-14d7c0337fff,BootID:6b9e799a-3b2a-44dc-8a4e-1a86f3b5230b,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 25 06:56:52.196: INFO:
Logging kubelet events for node i-0df245ff74f46fa48
Jan 25 06:56:52.230: INFO:
Logging pods the kubelet thinks is on node i-0df245ff74f46fa48
Jan 25 06:56:52.271: INFO: inline-volume-kh99b started at 2023-01-25 06:55:21 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:56:52.271: INFO: Container volume-tester ready: false, restart count 0
Jan 25 06:56:52.271: INFO: cilium-4kl5t started at 2023-01-25 06:27:28 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:56:52.271: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:56:52.271: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:56:52.271: INFO: ebs-csi-node-cfszw started at 2023-01-25 06:27:28 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:56:52.271: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:56:52.271: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:56:52.271: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:56:52.457: INFO:
Latency metrics for node i-0df245ff74f46fa48
Jan 25 06:56:52.457: INFO:
Logging node info for node i-0e8434bc9e1236fd5
Jan 25 06:56:52.489: INFO: Node Info: &Node{ObjectMeta:{i-0e8434bc9e1236fd5 48f670c2-fe1c-46d3-b197-464836dab89d 50816 0 2023-01-25 06:27:23 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0e8434bc9e1236fd5 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0e8434bc9e1236fd5 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.54.30 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7584":"i-0e8434bc9e1236fd5","ebs.csi.aws.com":"i-0e8434bc9e1236fd5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:55:19 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:55:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0e8434bc9e1236fd5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:55:19 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:55:19 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:55:19 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:55:19 +0000 UTC,LastTransitionTime:2023-01-25 06:27:35 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.30,},NodeAddress{Type:ExternalIP,Address:99.79.58.56,},NodeAddress{Type:InternalDNS,Address:i-0e8434bc9e1236fd5.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0e8434bc9e1236fd5.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-99-79-58-56.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2439e74c65fcfb2f8c6ece88db3944,SystemUUID:ec2439e7-4c65-fcfb-2f8c-6ece88db3944,BootID:568b3843-a4d7-46ef-b502-19f3e8310af2,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-7584^3869f6f7-9c7d-11ed-8a62-22a395ddc984],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-7584^3869f6f7-9c7d-11ed-8a62-22a395ddc984,DevicePath:,},},Config:nil,},}
Jan 25 06:56:52.489: INFO:
Logging kubelet events for node i-0e8434bc9e1236fd5
Jan 25 06:56:52.524: INFO:
Logging pods the kubelet thinks is on node i-0e8434bc9e1236fd5
Jan 25 06:56:52.563: INFO: coredns-autoscaler-7cb5c5b969-mndhj started at 2023-01-25 06:27:35 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:56:52.563: INFO: Container autoscaler ready: true, restart count 0
Jan 25 06:56:52.563: INFO: pvc-volume-tester-b6sqw started at 2023-01-25 06:55:17 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:56:52.563: INFO: Container volume-tester ready: true, restart count 0
Jan 25 06:56:52.563: INFO: csi-mockplugin-attacher-0 started at 2023-01-25 06:49:32 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:56:52.563: INFO: Container csi-attacher ready: true, restart count 0
Jan 25 06:56:52.563: INFO: cilium-xkk5h started at 2023-01-25 06:27:24 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:56:52.563: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:56:52.563: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:56:52.563: INFO: coredns-559769c974-ggjhw started at 2023-01-25 06:27:35 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:56:52.563: INFO: Container coredns ready: true, restart count 0
Jan 25 06:56:52.563: INFO: ebs-csi-node-4d5tv started at 2023-01-25 06:27:24 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:56:52.563: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:56:52.563: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:56:52.563: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:56:52.563: INFO: csi-mockplugin-0 started at 2023-01-25 06:49:32 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:56:52.563: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:56:52.563: INFO: Container driver-registrar ready: true, restart count 0
Jan 25 06:56:52.563: INFO: Container mock ready: true, restart count 0
Jan 25 06:56:52.795: INFO:
Latency metrics for node i-0e8434bc9e1236fd5
Jan 25 06:56:52.795: INFO:
Logging node info for node i-0ebbe4bd7ec58d213
Jan 25 06:56:52.826: INFO: Node Info: &Node{ObjectMeta:{i-0ebbe4bd7ec58d213 f69d5727-b51d-4595-ab9a-fe57b5c9c863 51146 0 2023-01-25 06:27:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0ebbe4bd7ec58d213 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0ebbe4bd7ec58d213 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.215 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-1869":"i-0ebbe4bd7ec58d213","ebs.csi.aws.com":"i-0ebbe4bd7ec58d213"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2023-01-25 06:56:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0ebbe4bd7ec58d213,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051685376 0} {<nil>} 3956724Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946827776 0} {<nil>} 3854324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:56:24 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:56:24 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:56:24 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:56:24 +0000 UTC,LastTransitionTime:2023-01-25 06:27:38 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.215,},NodeAddress{Type:ExternalIP,Address:3.96.126.35,},NodeAddress{Type:InternalDNS,Address:i-0ebbe4bd7ec58d213.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ebbe4bd7ec58d213.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-126-35.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2da1db9b923c887fdbe1dd4557a0d5,SystemUUID:ec2da1db-9b92-3c88-7fdb-e1dd4557a0d5,BootID:10184fa3-2e15-48fe-9718-382b9d68823e,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-1869^263a0900-9c7d-11ed-8b45-0ec750ba5342],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 25 06:56:52.827: INFO:
Logging kubelet events for node i-0ebbe4bd7ec58d213
Jan 25 06:56:52.861: INFO:
Logging pods the kubelet thinks is on node i-0ebbe4bd7ec58d213
Jan 25 06:56:52.895: INFO: ebs-csi-node-9ngsq started at 2023-01-25 06:27:27 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:56:52.895: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:56:52.895: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:56:52.895: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:56:52.895: INFO: coredns-559769c974-v87jf started at 2023-01-25 06:27:57 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:56:52.895: INFO: Container coredns ready: true, restart count 0
Jan 25 06:56:52.895: INFO: cilium-pkfh6 started at 2023-01-25 06:27:27 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:56:52.895: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:56:52.895: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:56:52.896: INFO: csi-hostpathplugin-0 started at 2023-01-25 06:49:47 +0000 UTC (0+7 container statuses recorded)
Jan 25 06:56:52.896: INFO: Container csi-attacher ready: true, restart count 0
Jan 25 06:56:52.896: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:56:52.896: INFO: Container csi-resizer ready: true, restart count 0
Jan 25 06:56:52.896: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 25 06:56:52.896: INFO: Container hostpath ready: true, restart count 0
Jan 25 06:56:52.896: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:56:52.896: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:56:53.103: INFO:
Latency metrics for node i-0ebbe4bd7ec58d213
�[1mSTEP:�[0m Waiting for namespaces [provisioning-1869] to vanish �[38;5;243m01/25/23 06:56:53.137�[0m
�[1mSTEP:�[0m uninstalling csi csi-hostpath driver �[38;5;243m01/25/23 06:57:05.17�[0m
Jan 25 06:57:05.170: INFO: deleting *v1.ServiceAccount: provisioning-1869-9392/csi-attacher
Jan 25 06:57:05.202: INFO: deleting *v1.ClusterRole: external-attacher-runner-provisioning-1869
Jan 25 06:57:05.235: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-provisioning-1869
Jan 25 06:57:05.267: INFO: deleting *v1.Role: provisioning-1869-9392/external-attacher-cfg-provisioning-1869
Jan 25 06:57:05.300: INFO: deleting *v1.RoleBinding: provisioning-1869-9392/csi-attacher-role-cfg
Jan 25 06:57:05.332: INFO: deleting *v1.ServiceAccount: provisioning-1869-9392/csi-provisioner
Jan 25 06:57:05.365: INFO: deleting *v1.ClusterRole: external-provisioner-runner-provisioning-1869
Jan 25 06:57:05.399: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-provisioning-1869
Jan 25 06:57:05.432: INFO: deleting *v1.Role: provisioning-1869-9392/external-provisioner-cfg-provisioning-1869
Jan 25 06:57:05.464: INFO: deleting *v1.RoleBinding: provisioning-1869-9392/csi-provisioner-role-cfg
Jan 25 06:57:05.497: INFO: deleting *v1.ServiceAccount: provisioning-1869-9392/csi-snapshotter
Jan 25 06:57:05.529: INFO: deleting *v1.ClusterRole: external-snapshotter-runner-provisioning-1869
Jan 25 06:57:05.562: INFO: deleting *v1.ClusterRoleBinding: csi-snapshotter-role-provisioning-1869
Jan 25 06:57:05.595: INFO: deleting *v1.Role: provisioning-1869-9392/external-snapshotter-leaderelection-provisioning-1869
Jan 25 06:57:05.628: INFO: deleting *v1.RoleBinding: provisioning-1869-9392/external-snapshotter-leaderelection
Jan 25 06:57:05.661: INFO: deleting *v1.ServiceAccount: provisioning-1869-9392/csi-external-health-monitor-controller
Jan 25 06:57:05.694: INFO: deleting *v1.ClusterRole: external-health-monitor-controller-runner-provisioning-1869
Jan 25 06:57:05.729: INFO: deleting *v1.ClusterRoleBinding: csi-external-health-monitor-controller-role-provisioning-1869
Jan 25 06:57:05.762: INFO: deleting *v1.Role: provisioning-1869-9392/external-health-monitor-controller-cfg-provisioning-1869
Jan 25 06:57:05.794: INFO: deleting *v1.RoleBinding: provisioning-1869-9392/csi-external-health-monitor-controller-role-cfg
Jan 25 06:57:05.827: INFO: deleting *v1.ServiceAccount: provisioning-1869-9392/csi-resizer
Jan 25 06:57:05.860: INFO: deleting *v1.ClusterRole: external-resizer-runner-provisioning-1869
Jan 25 06:57:05.893: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-provisioning-1869
Jan 25 06:57:05.925: INFO: deleting *v1.Role: provisioning-1869-9392/external-resizer-cfg-provisioning-1869
Jan 25 06:57:05.958: INFO: deleting *v1.RoleBinding: provisioning-1869-9392/csi-resizer-role-cfg
Jan 25 06:57:05.991: INFO: deleting *v1.CSIDriver: csi-hostpath-provisioning-1869
Jan 25 06:57:06.025: INFO: deleting *v1.ServiceAccount: provisioning-1869-9392/csi-hostpathplugin-sa
Jan 25 06:57:06.058: INFO: deleting *v1.ClusterRoleBinding: csi-hostpathplugin-attacher-cluster-role-provisioning-1869
Jan 25 06:57:06.091: INFO: deleting *v1.ClusterRoleBinding: csi-hostpathplugin-health-monitor-controller-cluster-role-provisioning-1869
Jan 25 06:57:06.123: INFO: deleting *v1.ClusterRoleBinding: csi-hostpathplugin-provisioner-cluster-role-provisioning-1869
Jan 25 06:57:06.156: INFO: deleting *v1.ClusterRoleBinding: csi-hostpathplugin-resizer-cluster-role-provisioning-1869
Jan 25 06:57:06.189: INFO: deleting *v1.ClusterRoleBinding: csi-hostpathplugin-snapshotter-cluster-role-provisioning-1869
Jan 25 06:57:06.222: INFO: deleting *v1.RoleBinding: provisioning-1869-9392/csi-hostpathplugin-attacher-role
Jan 25 06:57:06.255: INFO: deleting *v1.RoleBinding: provisioning-1869-9392/csi-hostpathplugin-health-monitor-controller-role
Jan 25 06:57:06.287: INFO: deleting *v1.RoleBinding: provisioning-1869-9392/csi-hostpathplugin-provisioner-role
Jan 25 06:57:06.321: INFO: deleting *v1.RoleBinding: provisioning-1869-9392/csi-hostpathplugin-resizer-role
Jan 25 06:57:06.354: INFO: deleting *v1.RoleBinding: provisioning-1869-9392/csi-hostpathplugin-snapshotter-role
Jan 25 06:57:06.387: INFO: deleting *v1.StatefulSet: provisioning-1869-9392/csi-hostpathplugin
Jan 25 06:57:06.420: INFO: deleting *v1.ClusterRoleBinding: psp-csi-hostpath-role-provisioning-1869
�[1mSTEP:�[0m deleting the driver namespace: provisioning-1869-9392 �[38;5;243m01/25/23 06:57:06.452�[0m
�[1mSTEP:�[0m Collecting events from namespace "provisioning-1869-9392". �[38;5;243m01/25/23 06:57:06.452�[0m
�[1mSTEP:�[0m Found 15 events. �[38;5;243m01/25/23 06:57:06.504�[0m
Jan 25 06:57:06.504: INFO: At 2023-01-25 06:49:47 +0000 UTC - event for csi-hostpathplugin: {statefulset-controller } SuccessfulCreate: create Pod csi-hostpathplugin-0 in StatefulSet csi-hostpathplugin successful
Jan 25 06:57:06.504: INFO: At 2023-01-25 06:49:47 +0000 UTC - event for csi-hostpathplugin-0: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "506450f5736948a49393ca8399c38e7df9fdfe045391e24128b251832a636ea9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:57:06.504: INFO: At 2023-01-25 06:49:47 +0000 UTC - event for csi-hostpathplugin-0: {default-scheduler } Scheduled: Successfully assigned provisioning-1869-9392/csi-hostpathplugin-0 to i-0ebbe4bd7ec58d213
Jan 25 06:57:06.504: INFO: At 2023-01-25 06:49:58 +0000 UTC - event for csi-hostpathplugin-0: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "86cc17d4518a6c30f85fccced92c1674ab56d8b7310cace9e0926e5deb584e8e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:57:06.504: INFO: At 2023-01-25 06:50:10 +0000 UTC - event for csi-hostpathplugin-0: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b05427b5d1f17bf37735a15ddb3fb8e0b2d3e732465dfe06d9bbb566c830b805": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:57:06.504: INFO: At 2023-01-25 06:50:21 +0000 UTC - event for csi-hostpathplugin-0: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6535b2ca3794a351801d77ff3fb5fe9e7863ed49382b57ec45b7d07563230512": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:57:06.504: INFO: At 2023-01-25 06:50:35 +0000 UTC - event for csi-hostpathplugin-0: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a4001d261c354cb37e7f76a45c172b720dd58c91b7cac55a44d418282e4c6dfa": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:57:06.504: INFO: At 2023-01-25 06:50:47 +0000 UTC - event for csi-hostpathplugin-0: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "2e9b7c5f662cfa5d9271d5089736ba55bbc5f0424e0274d1b2a1955e0a78a646": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:57:06.504: INFO: At 2023-01-25 06:50:59 +0000 UTC - event for csi-hostpathplugin-0: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "dd8b512615122ef01da0dd75f5dc60a492cd53a1e8d080f924114dd15f220662": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:57:06.504: INFO: At 2023-01-25 06:51:15 +0000 UTC - event for csi-hostpathplugin-0: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e9df2db235393a6e2cd7bcfe750131edc5999c3827ed6968229452ae3f10caf3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:57:06.504: INFO: At 2023-01-25 06:51:25 +0000 UTC - event for csi-hostpathplugin-0: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ad64fd022a13ae13a5dcf5bc9c076ed2eb7177d3cdd3012e73b460e0b3216980": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:57:06.504: INFO: At 2023-01-25 06:51:40 +0000 UTC - event for csi-hostpathplugin-0: {kubelet i-0ebbe4bd7ec58d213} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c88e732b5ce861380279af595502fdd4defc373e8215cdd623f99d445c7ade44": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:57:06.504: INFO: At 2023-01-25 06:54:43 +0000 UTC - event for csi-hostpathplugin-0: {kubelet i-0ebbe4bd7ec58d213} Pulled: Container image "registry.k8s.io/sig-storage/hostpathplugin:v1.9.0" already present on machine
Jan 25 06:57:06.504: INFO: At 2023-01-25 06:54:43 +0000 UTC - event for csi-hostpathplugin-0: {kubelet i-0ebbe4bd7ec58d213} Created: Created container hostpath
Jan 25 06:57:06.504: INFO: At 2023-01-25 06:57:06 +0000 UTC - event for csi-hostpathplugin-0: {kubelet i-0ebbe4bd7ec58d213} Killing: Stopping container hostpath
Jan 25 06:57:06.535: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 25 06:57:06.535: INFO: csi-hostpathplugin-0 i-0ebbe4bd7ec58d213 Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:49:47 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:54:44 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:54:44 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-25 06:49:47 +0000 UTC }]
Jan 25 06:57:06.535: INFO:
Jan 25 06:57:06.898: INFO:
Logging node info for node i-071cd767a01ed5a98
Jan 25 06:57:06.930: INFO: Node Info: &Node{ObjectMeta:{i-071cd767a01ed5a98 e176a57e-e197-4621-9ff9-1551f1909075 49349 0 2023-01-25 06:25:20 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-071cd767a01ed5a98 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ca-central-1a topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.39.108 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-071cd767a01ed5a98"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-25 06:25:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-25 06:25:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:26:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:26:12 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-25 06:26:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-25 06:53:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-071cd767a01ed5a98,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862937600 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758080000 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:53:16 +0000 UTC,LastTransitionTime:2023-01-25 06:26:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.108,},NodeAddress{Type:ExternalIP,Address:3.97.7.38,},NodeAddress{Type:InternalDNS,Address:i-071cd767a01ed5a98.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-071cd767a01ed5a98.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-97-7-38.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dc6c1622927717fff709d069d68b2,SystemUUID:ec2dc6c1-6229-2771-7fff-709d069d68b2,BootID:034bcfe8-9462-416d-9a80-bb9dcb9fd35d,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.1],SizeBytes:135178704,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.1],SizeBytes:124995897,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:63920476,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.1],SizeBytes:57661752,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:44339025,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42841295,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4965790,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 25 06:57:06.930: INFO:
Logging kubelet events for node i-071cd767a01ed5a98
Jan 25 06:57:06.964: INFO:
Logging pods the kubelet thinks is on node i-071cd767a01ed5a98
Jan 25 06:57:07.002: INFO: cilium-nqkjx started at 2023-01-25 06:26:00 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:57:07.002: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:57:07.002: INFO: dns-controller-69987775c6-7nx57 started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:57:07.002: INFO: Container dns-controller ready: true, restart count 0
Jan 25 06:57:07.002: INFO: aws-cloud-controller-manager-6kpfv started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:57:07.002: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 25 06:57:07.002: INFO: etcd-manager-events-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (10+1 container statuses recorded)
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Container etcd-manager ready: true, restart count 0
Jan 25 06:57:07.002: INFO: etcd-manager-main-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (10+1 container statuses recorded)
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Container etcd-manager ready: true, restart count 0
Jan 25 06:57:07.002: INFO: kube-apiserver-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+2 container statuses recorded)
Jan 25 06:57:07.002: INFO: Container healthcheck ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Container kube-apiserver ready: true, restart count 3
Jan 25 06:57:07.002: INFO: kube-controller-manager-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:57:07.002: INFO: Container kube-controller-manager ready: true, restart count 4
Jan 25 06:57:07.002: INFO: kube-scheduler-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:57:07.002: INFO: Container kube-scheduler ready: true, restart count 0
Jan 25 06:57:07.002: INFO: kops-controller-qfzp7 started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:57:07.002: INFO: Container kops-controller ready: true, restart count 0
Jan 25 06:57:07.002: INFO: ebs-csi-controller-5bd98b456f-cdjzf started at 2023-01-25 06:26:00 +0000 UTC (0+5 container statuses recorded)
Jan 25 06:57:07.002: INFO: Container csi-attacher ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Container csi-resizer ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:57:07.002: INFO: ebs-csi-node-7t2hw started at 2023-01-25 06:26:00 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:57:07.002: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:57:07.002: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:57:07.002: INFO: cilium-operator-5dd44dc49f-jhvqd started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:57:07.002: INFO: Container cilium-operator ready: true, restart count 0
Jan 25 06:57:07.157: INFO:
Latency metrics for node i-071cd767a01ed5a98
Jan 25 06:57:07.157: INFO:
Logging node info for node i-0dbcb4dc9e1f5e62b
Jan 25 06:57:07.188: INFO: Node Info: &Node{ObjectMeta:{i-0dbcb4dc9e1f5e62b f1ad0cae-2ed9-41a4-8be5-9884ae32adaa 51044 0 2023-01-25 06:27:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0dbcb4dc9e1f5e62b kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0dbcb4dc9e1f5e62b topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.63.11 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-9383":"i-0dbcb4dc9e1f5e62b","ebs.csi.aws.com":"i-0dbcb4dc9e1f5e62b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:55:59 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:56:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0dbcb4dc9e1f5e62b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:56:00 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:56:00 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:56:00 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:56:00 +0000 UTC,LastTransitionTime:2023-01-25 06:27:38 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.11,},NodeAddress{Type:ExternalIP,Address:3.96.138.144,},NodeAddress{Type:InternalDNS,Address:i-0dbcb4dc9e1f5e62b.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0dbcb4dc9e1f5e62b.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-138-144.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2cc8535b4596aa012ef7857efd2302,SystemUUID:ec2cc853-5b45-96aa-012e-f7857efd2302,BootID:1d85d5b3-ad1c-462b-98e2-f01921a4e990,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-9383^5258fd8d-9c7d-11ed-aea5-a678623d062f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-9383^5258fd8d-9c7d-11ed-aea5-a678623d062f,DevicePath:,},},Config:nil,},}
Jan 25 06:57:07.189: INFO:
Logging kubelet events for node i-0dbcb4dc9e1f5e62b
Jan 25 06:57:07.223: INFO:
Logging pods the kubelet thinks is on node i-0dbcb4dc9e1f5e62b
Jan 25 06:57:07.258: INFO: ebs-csi-node-bn5vq started at 2023-01-25 06:27:26 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:57:07.258: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:57:07.258: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:57:07.258: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:57:07.258: INFO: cilium-wptjp started at 2023-01-25 06:27:26 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:57:07.258: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:57:07.258: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:57:07.258: INFO: csi-mockplugin-attacher-0 started at 2023-01-25 06:50:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:57:07.258: INFO: Container csi-attacher ready: true, restart count 0
Jan 25 06:57:07.258: INFO: csi-mockplugin-0 started at 2023-01-25 06:50:15 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:57:07.258: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:57:07.258: INFO: Container driver-registrar ready: true, restart count 0
Jan 25 06:57:07.258: INFO: Container mock ready: true, restart count 0
Jan 25 06:57:07.258: INFO: csi-mockplugin-resizer-0 started at 2023-01-25 06:50:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:57:07.258: INFO: Container csi-resizer ready: true, restart count 0
Jan 25 06:57:07.258: INFO: pvc-volume-tester-rx8qh started at 2023-01-25 06:55:58 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:57:07.258: INFO: Container volume-tester ready: true, restart count 0
Jan 25 06:57:07.458: INFO:
Latency metrics for node i-0dbcb4dc9e1f5e62b
Jan 25 06:57:07.458: INFO:
Logging node info for node i-0df245ff74f46fa48
Jan 25 06:57:07.489: INFO: Node Info: &Node{ObjectMeta:{i-0df245ff74f46fa48 3893284f-3580-49f6-b7a1-2ddba9ae06ec 50214 0 2023-01-25 06:27:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0df245ff74f46fa48 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0df245ff74f46fa48 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.41.87 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0df245ff74f46fa48"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kops-controller Update v1 2023-01-25 06:27:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-25 06:54:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0df245ff74f46fa48,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:54:52 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:54:52 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:54:52 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:54:52 +0000 UTC,LastTransitionTime:2023-01-25 06:27:40 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.87,},NodeAddress{Type:ExternalIP,Address:3.96.184.142,},NodeAddress{Type:InternalDNS,Address:i-0df245ff74f46fa48.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0df245ff74f46fa48.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-184-142.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2129fadd2b2b52f98b14d7c0337fff,SystemUUID:ec2129fa-dd2b-2b52-f98b-14d7c0337fff,BootID:6b9e799a-3b2a-44dc-8a4e-1a86f3b5230b,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 25 06:57:07.489: INFO:
Logging kubelet events for node i-0df245ff74f46fa48
Jan 25 06:57:07.523: INFO:
Logging pods the kubelet thinks is on node i-0df245ff74f46fa48
Jan 25 06:57:07.559: INFO: cilium-4kl5t started at 2023-01-25 06:27:28 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:57:07.559: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:57:07.559: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:57:07.559: INFO: inline-volume-kh99b started at 2023-01-25 06:55:21 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:57:07.559: INFO: Container volume-tester ready: false, restart count 0
Jan 25 06:57:07.559: INFO: ebs-csi-node-cfszw started at 2023-01-25 06:27:28 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:57:07.559: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:57:07.559: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:57:07.559: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:57:07.743: INFO:
Latency metrics for node i-0df245ff74f46fa48
Jan 25 06:57:07.743: INFO:
Logging node info for node i-0e8434bc9e1236fd5
Jan 25 06:57:07.774: INFO: Node Info: &Node{ObjectMeta:{i-0e8434bc9e1236fd5 48f670c2-fe1c-46d3-b197-464836dab89d 50816 0 2023-01-25 06:27:23 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0e8434bc9e1236fd5 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0e8434bc9e1236fd5 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.54.30 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7584":"i-0e8434bc9e1236fd5","ebs.csi.aws.com":"i-0e8434bc9e1236fd5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:55:19 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:55:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0e8434bc9e1236fd5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:55:19 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:55:19 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:55:19 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:55:19 +0000 UTC,LastTransitionTime:2023-01-25 06:27:35 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.30,},NodeAddress{Type:ExternalIP,Address:99.79.58.56,},NodeAddress{Type:InternalDNS,Address:i-0e8434bc9e1236fd5.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0e8434bc9e1236fd5.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-99-79-58-56.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2439e74c65fcfb2f8c6ece88db3944,SystemUUID:ec2439e7-4c65-fcfb-2f8c-6ece88db3944,BootID:568b3843-a4d7-46ef-b502-19f3e8310af2,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-7584^3869f6f7-9c7d-11ed-8a62-22a395ddc984],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-7584^3869f6f7-9c7d-11ed-8a62-22a395ddc984,DevicePath:,},},Config:nil,},}
Jan 25 06:57:07.775: INFO:
Logging kubelet events for node i-0e8434bc9e1236fd5
Jan 25 06:57:07.809: INFO:
Logging pods the kubelet thinks is on node i-0e8434bc9e1236fd5
Jan 25 06:57:07.844: INFO: ebs-csi-node-4d5tv started at 2023-01-25 06:27:24 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:57:07.844: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:57:07.844: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:57:07.844: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:57:07.844: INFO: csi-mockplugin-0 started at 2023-01-25 06:49:32 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:57:07.844: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:57:07.844: INFO: Container driver-registrar ready: true, restart count 0
Jan 25 06:57:07.844: INFO: Container mock ready: true, restart count 0
Jan 25 06:57:07.844: INFO: coredns-autoscaler-7cb5c5b969-mndhj started at 2023-01-25 06:27:35 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:57:07.844: INFO: Container autoscaler ready: true, restart count 0
Jan 25 06:57:07.844: INFO: pvc-volume-tester-b6sqw started at 2023-01-25 06:55:17 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:57:07.844: INFO: Container volume-tester ready: true, restart count 0
Jan 25 06:57:07.844: INFO: csi-mockplugin-attacher-0 started at 2023-01-25 06:49:32 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:57:07.844: INFO: Container csi-attacher ready: true, restart count 0
Jan 25 06:57:07.844: INFO: cilium-xkk5h started at 2023-01-25 06:27:24 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:57:07.844: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:57:07.844: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:57:07.844: INFO: coredns-559769c974-ggjhw started at 2023-01-25 06:27:35 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:57:07.844: INFO: Container coredns ready: true, restart count 0
Jan 25 06:57:08.022: INFO:
Latency metrics for node i-0e8434bc9e1236fd5
Jan 25 06:57:08.022: INFO:
Logging node info for node i-0ebbe4bd7ec58d213
Jan 25 06:57:08.054: INFO: Node Info: &Node{ObjectMeta:{i-0ebbe4bd7ec58d213 f69d5727-b51d-4595-ab9a-fe57b5c9c863 51369 0 2023-01-25 06:27:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0ebbe4bd7ec58d213 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0ebbe4bd7ec58d213 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.215 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0ebbe4bd7ec58d213"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2023-01-25 06:57:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0ebbe4bd7ec58d213,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051685376 0} {<nil>} 3956724Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946827776 0} {<nil>} 3854324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:56:54 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:56:54 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:56:54 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:56:54 +0000 UTC,LastTransitionTime:2023-01-25 06:27:38 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.215,},NodeAddress{Type:ExternalIP,Address:3.96.126.35,},NodeAddress{Type:InternalDNS,Address:i-0ebbe4bd7ec58d213.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ebbe4bd7ec58d213.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-126-35.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2da1db9b923c887fdbe1dd4557a0d5,SystemUUID:ec2da1db-9b92-3c88-7fdb-e1dd4557a0d5,BootID:10184fa3-2e15-48fe-9718-382b9d68823e,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 25 06:57:08.054: INFO:
Logging kubelet events for node i-0ebbe4bd7ec58d213
Jan 25 06:57:08.090: INFO:
Logging pods the kubelet thinks is on node i-0ebbe4bd7ec58d213
Jan 25 06:57:08.125: INFO: ebs-csi-node-9ngsq started at 2023-01-25 06:27:27 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:57:08.125: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:57:08.125: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:57:08.125: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:57:08.125: INFO: cilium-pkfh6 started at 2023-01-25 06:27:27 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:57:08.125: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:57:08.125: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:57:08.125: INFO: coredns-559769c974-v87jf started at 2023-01-25 06:27:57 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:57:08.125: INFO: Container coredns ready: true, restart count 0
Jan 25 06:57:08.298: INFO:
Latency metrics for node i-0ebbe4bd7ec58d213
�[1mSTEP:�[0m Waiting for namespaces [provisioning-1869-9392] to vanish �[38;5;243m01/25/23 06:57:08.331�[0m
[DeferCleanup (Each)] [Testpattern: Dynamic PV (block volmode)] provisioning
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [Testpattern: Dynamic PV (block volmode)] provisioning
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/25/23 06:57:14.364�[0m
[DeferCleanup (Each)] [Testpattern: Dynamic PV (block volmode)] provisioning
tear down framework | framework.go:193
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-storage\]\sIn\-tree\sVolumes\s\[Driver\:\slocal\]\[LocalVolumeType\:\sdir\-link\]\s\[Testpattern\:\sPre\-provisioned\sPV\s\(default\sfs\)\]\ssubPath\sshould\ssupport\sexisting\sdirectories\swhen\sreadOnly\sspecified\sin\sthe\svolumeSource$'
test/e2e/framework/pod/output/output.go:237
k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutputMatcher(0xc0030b9200?, {0x75c4bbe?, 0x0?}, 0xc0030b9200, 0x0, {0xc0006f7e70, 0x1, 0x1}, 0x22?)
test/e2e/framework/pod/output/output.go:237 +0x176
k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutput(...)
test/e2e/framework/pod/output/output.go:214
k8s.io/kubernetes/test/e2e/storage/testsuites.testReadFile(0xc0011c25a0, {0xc003dea258, 0x16}, 0xc0030b9200, 0x0)
test/e2e/storage/testsuites/subpath.go:691 +0x15c
k8s.io/kubernetes/test/e2e/storage/testsuites.(*subPathTestSuite).DefineTests.func18()
test/e2e/storage/testsuites/subpath.go:411 +0x24f
[BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath
test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/25/23 06:46:43.835�[0m
Jan 25 06:46:43.835: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename provisioning �[38;5;243m01/25/23 06:46:43.837�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/25/23 06:46:43.931�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/25/23 06:46:43.992�[0m
[BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath
test/e2e/framework/metrics/init/init.go:31
[It] should support existing directories when readOnly specified in the volumeSource
test/e2e/storage/testsuites/subpath.go:395
Jan 25 06:46:44.085: INFO: In-tree plugin kubernetes.io/local-volume is not migrated, not validating any metrics
Jan 25 06:46:44.120: INFO: Waiting up to 5m0s for pod "hostexec-i-0df245ff74f46fa48-xd5sx" in namespace "provisioning-2863" to be "running"
Jan 25 06:46:44.151: INFO: Pod "hostexec-i-0df245ff74f46fa48-xd5sx": Phase="Pending", Reason="", readiness=false. Elapsed: 30.819375ms
Jan 25 06:46:46.182: INFO: Pod "hostexec-i-0df245ff74f46fa48-xd5sx": Phase="Running", Reason="", readiness=true. Elapsed: 2.062288196s
Jan 25 06:46:46.182: INFO: Pod "hostexec-i-0df245ff74f46fa48-xd5sx" satisfied condition "running"
Jan 25 06:46:46.182: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-driver-06061841-7a10-4ce5-9316-770a8615d7ee-backend && ln -s /tmp/local-driver-06061841-7a10-4ce5-9316-770a8615d7ee-backend /tmp/local-driver-06061841-7a10-4ce5-9316-770a8615d7ee] Namespace:provisioning-2863 PodName:hostexec-i-0df245ff74f46fa48-xd5sx ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true Quiet:false}
Jan 25 06:46:46.182: INFO: >>> kubeConfig: /root/.kube/config
Jan 25 06:46:46.183: INFO: ExecWithOptions: Clientset creation
Jan 25 06:46:46.183: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-2863/pods/hostexec-i-0df245ff74f46fa48-xd5sx/exec?command=nsenter&command=--mount%3D%2Frootfs%2Fproc%2F1%2Fns%2Fmnt&command=--&command=sh&command=-c&command=mkdir+%2Ftmp%2Flocal-driver-06061841-7a10-4ce5-9316-770a8615d7ee-backend+%26%26+ln+-s+%2Ftmp%2Flocal-driver-06061841-7a10-4ce5-9316-770a8615d7ee-backend+%2Ftmp%2Flocal-driver-06061841-7a10-4ce5-9316-770a8615d7ee&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true)
Jan 25 06:46:46.456: INFO: Creating resource for pre-provisioned PV
Jan 25 06:46:46.456: INFO: Creating PVC and PV
�[1mSTEP:�[0m Creating a PVC followed by a PV �[38;5;243m01/25/23 06:46:46.456�[0m
Jan 25 06:46:46.522: INFO: Waiting for PV local-zfsgt to bind to PVC pvc-gzk2j
Jan 25 06:46:46.522: INFO: Waiting up to timeout=3m0s for PersistentVolumeClaims [pvc-gzk2j] to have phase Bound
Jan 25 06:46:46.553: INFO: PersistentVolumeClaim pvc-gzk2j found but phase is Pending instead of Bound.
Jan 25 06:46:48.585: INFO: PersistentVolumeClaim pvc-gzk2j found but phase is Pending instead of Bound.
Jan 25 06:46:50.616: INFO: PersistentVolumeClaim pvc-gzk2j found and phase=Bound (4.094377304s)
Jan 25 06:46:50.616: INFO: Waiting up to 3m0s for PersistentVolume local-zfsgt to have phase Bound
Jan 25 06:46:50.647: INFO: PersistentVolume local-zfsgt found and phase=Bound (30.770081ms)
�[1mSTEP:�[0m Creating pod pod-subpath-test-preprovisionedpv-7rxj �[38;5;243m01/25/23 06:46:50.709�[0m
�[1mSTEP:�[0m Creating a pod to test subpath �[38;5;243m01/25/23 06:46:50.709�[0m
Jan 25 06:46:50.743: INFO: Waiting up to 5m0s for pod "pod-subpath-test-preprovisionedpv-7rxj" in namespace "provisioning-2863" to be "Succeeded or Failed"
Jan 25 06:46:50.774: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 30.98054ms
Jan 25 06:46:52.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2.063207937s
Jan 25 06:46:54.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4.062781183s
Jan 25 06:46:56.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 6.06337159s
Jan 25 06:46:58.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 8.063449898s
Jan 25 06:47:00.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 10.063142387s
Jan 25 06:47:02.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 12.063218381s
Jan 25 06:47:04.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 14.062761783s
Jan 25 06:47:06.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 16.062887226s
Jan 25 06:47:08.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 18.063252038s
Jan 25 06:47:10.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 20.063137098s
Jan 25 06:47:12.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 22.063245818s
Jan 25 06:47:14.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 24.062690454s
Jan 25 06:47:16.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 26.062707658s
Jan 25 06:47:18.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 28.06318369s
Jan 25 06:47:20.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 30.06346094s
Jan 25 06:47:22.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 32.063305768s
Jan 25 06:47:24.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 34.063337264s
Jan 25 06:47:26.812: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 36.068187268s
Jan 25 06:47:28.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 38.062502352s
Jan 25 06:47:30.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 40.063282744s
Jan 25 06:47:32.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 42.063411238s
Jan 25 06:47:34.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 44.063099862s
Jan 25 06:47:36.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 46.063402257s
Jan 25 06:47:38.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 48.063530768s
Jan 25 06:47:40.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 50.06229085s
Jan 25 06:47:42.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 52.062781622s
Jan 25 06:47:44.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 54.06290275s
Jan 25 06:47:46.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 56.063493948s
Jan 25 06:47:48.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 58.063118918s
Jan 25 06:47:50.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.062925479s
Jan 25 06:47:52.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.062834788s
Jan 25 06:47:54.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.062982899s
Jan 25 06:47:56.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.063136321s
Jan 25 06:47:58.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.063113748s
Jan 25 06:48:00.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.063152517s
Jan 25 06:48:02.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.062912592s
Jan 25 06:48:04.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.063247533s
Jan 25 06:48:06.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.062996496s
Jan 25 06:48:08.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.063253079s
Jan 25 06:48:10.808: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.065046121s
Jan 25 06:48:12.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.063078521s
Jan 25 06:48:14.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.06275135s
Jan 25 06:48:16.809: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.065222563s
Jan 25 06:48:18.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.063002376s
Jan 25 06:48:20.814: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.070185998s
Jan 25 06:48:22.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.062846483s
Jan 25 06:48:24.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.062910021s
Jan 25 06:48:26.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.063159025s
Jan 25 06:48:28.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.063247711s
Jan 25 06:48:30.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.063038421s
Jan 25 06:48:32.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.062784743s
Jan 25 06:48:34.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.06277919s
Jan 25 06:48:36.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.063008247s
Jan 25 06:48:38.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.063449246s
Jan 25 06:48:40.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.062757079s
Jan 25 06:48:42.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.063665315s
Jan 25 06:48:44.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.062999172s
Jan 25 06:48:46.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.063406169s
Jan 25 06:48:48.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.062652412s
Jan 25 06:48:50.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.063063297s
Jan 25 06:48:52.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.063667112s
Jan 25 06:48:54.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.063152507s
Jan 25 06:48:56.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.063252316s
Jan 25 06:48:58.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.063359499s
Jan 25 06:49:00.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.062936975s
Jan 25 06:49:02.810: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.066771652s
Jan 25 06:49:04.808: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.064794514s
Jan 25 06:49:06.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.062817693s
Jan 25 06:49:08.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.063083563s
Jan 25 06:49:10.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.062279408s
Jan 25 06:49:12.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.064011955s
Jan 25 06:49:14.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.062993204s
Jan 25 06:49:16.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.062627934s
Jan 25 06:49:18.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.063501s
Jan 25 06:49:20.808: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.065136672s
Jan 25 06:49:22.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.063447043s
Jan 25 06:49:24.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.063259722s
Jan 25 06:49:26.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.064102863s
Jan 25 06:49:28.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.063890568s
Jan 25 06:49:30.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.063187708s
Jan 25 06:49:32.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.063046914s
Jan 25 06:49:34.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.062536438s
Jan 25 06:49:36.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.063454316s
Jan 25 06:49:38.811: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.067267871s
Jan 25 06:49:40.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.063243449s
Jan 25 06:49:42.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.063535632s
Jan 25 06:49:44.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.06321162s
Jan 25 06:49:46.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.06325786s
Jan 25 06:49:48.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.063020129s
Jan 25 06:49:50.808: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.064576114s
Jan 25 06:49:52.808: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.064654838s
Jan 25 06:49:54.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.06298477s
Jan 25 06:49:56.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.062723769s
Jan 25 06:49:58.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.062971668s
Jan 25 06:50:00.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.062880234s
Jan 25 06:50:02.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.063232369s
Jan 25 06:50:04.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.063633904s
Jan 25 06:50:06.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.063163076s
Jan 25 06:50:08.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.063115175s
Jan 25 06:50:10.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.06310029s
Jan 25 06:50:12.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.062876298s
Jan 25 06:50:14.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.06306688s
Jan 25 06:50:16.827: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.08411758s
Jan 25 06:50:18.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.063410133s
Jan 25 06:50:20.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.062837383s
Jan 25 06:50:22.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.063172985s
Jan 25 06:50:24.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.062865824s
Jan 25 06:50:26.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.062982328s
Jan 25 06:50:28.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.063009831s
Jan 25 06:50:30.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.062828179s
Jan 25 06:50:32.810: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.066265622s
Jan 25 06:50:34.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.063159487s
Jan 25 06:50:36.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.063196076s
Jan 25 06:50:38.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.063265317s
Jan 25 06:50:40.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.062832126s
Jan 25 06:50:42.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.062984013s
Jan 25 06:50:44.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.06258923s
Jan 25 06:50:46.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.062873517s
Jan 25 06:50:48.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.06309077s
Jan 25 06:50:50.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.063141281s
Jan 25 06:50:52.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.062449565s
Jan 25 06:50:54.809: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.065262955s
Jan 25 06:50:56.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.062996572s
Jan 25 06:50:58.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.062902491s
Jan 25 06:51:00.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.063279963s
Jan 25 06:51:02.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.063258936s
Jan 25 06:51:04.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.063331947s
Jan 25 06:51:06.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.063109368s
Jan 25 06:51:08.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.063410107s
Jan 25 06:51:10.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.06297697s
Jan 25 06:51:12.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.062799225s
Jan 25 06:51:14.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.062647986s
Jan 25 06:51:16.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.06309858s
Jan 25 06:51:18.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.06300424s
Jan 25 06:51:20.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.063812617s
Jan 25 06:51:22.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.063234555s
Jan 25 06:51:24.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.062569061s
Jan 25 06:51:26.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.063084955s
Jan 25 06:51:28.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.063290143s
Jan 25 06:51:30.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.063791412s
Jan 25 06:51:32.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.062892012s
Jan 25 06:51:34.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.063041675s
Jan 25 06:51:36.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.0632466s
Jan 25 06:51:38.811: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.067226667s
Jan 25 06:51:40.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.063175642s
Jan 25 06:51:42.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.063207661s
Jan 25 06:51:44.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.06307135s
Jan 25 06:51:46.806: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.06220485s
Jan 25 06:51:48.810: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.066357449s
Jan 25 06:51:50.807: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.063287935s
Jan 25 06:51:50.838: INFO: Pod "pod-subpath-test-preprovisionedpv-7rxj": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.094776246s
Jan 25 06:51:50.902: INFO: Failed to get logs from node "i-0df245ff74f46fa48" pod "pod-subpath-test-preprovisionedpv-7rxj" container "init-volume-preprovisionedpv-7rxj": the server rejected our request for an unknown reason (get pods pod-subpath-test-preprovisionedpv-7rxj)
Jan 25 06:51:50.936: INFO: Failed to get logs from node "i-0df245ff74f46fa48" pod "pod-subpath-test-preprovisionedpv-7rxj" container "test-init-subpath-preprovisionedpv-7rxj": the server rejected our request for an unknown reason (get pods pod-subpath-test-preprovisionedpv-7rxj)
Jan 25 06:51:50.969: INFO: Failed to get logs from node "i-0df245ff74f46fa48" pod "pod-subpath-test-preprovisionedpv-7rxj" container "test-container-subpath-preprovisionedpv-7rxj": the server rejected our request for an unknown reason (get pods pod-subpath-test-preprovisionedpv-7rxj)
�[1mSTEP:�[0m delete the pod �[38;5;243m01/25/23 06:51:50.969�[0m
Jan 25 06:51:51.003: INFO: Waiting for pod pod-subpath-test-preprovisionedpv-7rxj to disappear
Jan 25 06:51:51.035: INFO: Pod pod-subpath-test-preprovisionedpv-7rxj still exists
Jan 25 06:51:53.035: INFO: Waiting for pod pod-subpath-test-preprovisionedpv-7rxj to disappear
Jan 25 06:51:53.066: INFO: Pod pod-subpath-test-preprovisionedpv-7rxj no longer exists
Jan 25 06:51:53.067: INFO: Unexpected error:
<*errors.errorString | 0xc004f9daa0>: {
s: "expected pod \"pod-subpath-test-preprovisionedpv-7rxj\" success: timed out while waiting for pod provisioning-2863/pod-subpath-test-preprovisionedpv-7rxj to be Succeeded or Failed",
}
Jan 25 06:51:53.067: FAIL: expected pod "pod-subpath-test-preprovisionedpv-7rxj" success: timed out while waiting for pod provisioning-2863/pod-subpath-test-preprovisionedpv-7rxj to be Succeeded or Failed
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutputMatcher(0xc0030b9200?, {0x75c4bbe?, 0x0?}, 0xc0030b9200, 0x0, {0xc0006f7e70, 0x1, 0x1}, 0x22?)
test/e2e/framework/pod/output/output.go:237 +0x176
k8s.io/kubernetes/test/e2e/framework/pod/output.TestContainerOutput(...)
test/e2e/framework/pod/output/output.go:214
k8s.io/kubernetes/test/e2e/storage/testsuites.testReadFile(0xc0011c25a0, {0xc003dea258, 0x16}, 0xc0030b9200, 0x0)
test/e2e/storage/testsuites/subpath.go:691 +0x15c
k8s.io/kubernetes/test/e2e/storage/testsuites.(*subPathTestSuite).DefineTests.func18()
test/e2e/storage/testsuites/subpath.go:411 +0x24f
�[1mSTEP:�[0m Deleting pod �[38;5;243m01/25/23 06:51:53.067�[0m
Jan 25 06:51:53.067: INFO: Deleting pod "pod-subpath-test-preprovisionedpv-7rxj" in namespace "provisioning-2863"
�[1mSTEP:�[0m Deleting pv and pvc �[38;5;243m01/25/23 06:51:53.098�[0m
Jan 25 06:51:53.098: INFO: Deleting PersistentVolumeClaim "pvc-gzk2j"
Jan 25 06:51:53.131: INFO: Deleting PersistentVolume "local-zfsgt"
�[1mSTEP:�[0m Removing the test directory �[38;5;243m01/25/23 06:51:53.164�[0m
Jan 25 06:51:53.164: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -r /tmp/local-driver-06061841-7a10-4ce5-9316-770a8615d7ee && rm -r /tmp/local-driver-06061841-7a10-4ce5-9316-770a8615d7ee-backend] Namespace:provisioning-2863 PodName:hostexec-i-0df245ff74f46fa48-xd5sx ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true Quiet:false}
Jan 25 06:51:53.164: INFO: >>> kubeConfig: /root/.kube/config
Jan 25 06:51:53.165: INFO: ExecWithOptions: Clientset creation
Jan 25 06:51:53.165: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/provisioning-2863/pods/hostexec-i-0df245ff74f46fa48-xd5sx/exec?command=nsenter&command=--mount%3D%2Frootfs%2Fproc%2F1%2Fns%2Fmnt&command=--&command=sh&command=-c&command=rm+-r+%2Ftmp%2Flocal-driver-06061841-7a10-4ce5-9316-770a8615d7ee+%26%26+rm+-r+%2Ftmp%2Flocal-driver-06061841-7a10-4ce5-9316-770a8615d7ee-backend&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true)
[AfterEach] [Testpattern: Pre-provisioned PV (default fs)] subPath
test/e2e/framework/node/init/init.go:32
Jan 25 06:51:53.452: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath
test/e2e/storage/drivers/in_tree.go:1734
�[1mSTEP:�[0m Deleting pod hostexec-i-0df245ff74f46fa48-xd5sx in namespace provisioning-2863 �[38;5;243m01/25/23 06:51:53.485�[0m
[DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/25/23 06:51:53.526�[0m
�[1mSTEP:�[0m Collecting events from namespace "provisioning-2863". �[38;5;243m01/25/23 06:51:53.526�[0m
�[1mSTEP:�[0m Found 17 events. �[38;5;243m01/25/23 06:51:53.558�[0m
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:46:44 +0000 UTC - event for hostexec-i-0df245ff74f46fa48-xd5sx: {default-scheduler } Scheduled: Successfully assigned provisioning-2863/hostexec-i-0df245ff74f46fa48-xd5sx to i-0df245ff74f46fa48
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:46:44 +0000 UTC - event for hostexec-i-0df245ff74f46fa48-xd5sx: {kubelet i-0df245ff74f46fa48} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:46:44 +0000 UTC - event for hostexec-i-0df245ff74f46fa48-xd5sx: {kubelet i-0df245ff74f46fa48} Created: Created container agnhost-container
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:46:44 +0000 UTC - event for hostexec-i-0df245ff74f46fa48-xd5sx: {kubelet i-0df245ff74f46fa48} Started: Started container agnhost-container
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:46:46 +0000 UTC - event for pvc-gzk2j: {persistentvolume-controller } ProvisioningFailed: storageclass.storage.k8s.io "provisioning-2863" not found
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:46:50 +0000 UTC - event for pod-subpath-test-preprovisionedpv-7rxj: {default-scheduler } Scheduled: Successfully assigned provisioning-2863/pod-subpath-test-preprovisionedpv-7rxj to i-0df245ff74f46fa48
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:46:51 +0000 UTC - event for pod-subpath-test-preprovisionedpv-7rxj: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cc494afa79012ee324351332656eb14159904068f22daaaba570027de05bd3bd": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:47:02 +0000 UTC - event for pod-subpath-test-preprovisionedpv-7rxj: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8a5c11b8408a8df93b2cfde505f608c96f27fe67af0e3cb020c2c9a698004ab1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:47:14 +0000 UTC - event for pod-subpath-test-preprovisionedpv-7rxj: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4892958c6f9bd10b382fcfa7830a20f370dcc40a8d203dde8f1818d7a1713152": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:47:25 +0000 UTC - event for pod-subpath-test-preprovisionedpv-7rxj: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "39fc7afa864f27bb4b3fe8162b45c8e0c9abc9c1e6c51caec260638ff68100d7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:47:37 +0000 UTC - event for pod-subpath-test-preprovisionedpv-7rxj: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ddf1bb5ed7ddb14a791f0a4fa729d4799c697fb7a0ba194d4f9b56f1e6888099": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:47:52 +0000 UTC - event for pod-subpath-test-preprovisionedpv-7rxj: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "900b86e21062e47f701b3f1ee914343af9b9fae95102564fb5d4ed9e67f4bb74": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:48:04 +0000 UTC - event for pod-subpath-test-preprovisionedpv-7rxj: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bbd24e56d48874df24823ad5e6fd59770fa7dcdb5563be537c1e90eaf786f9d6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:48:18 +0000 UTC - event for pod-subpath-test-preprovisionedpv-7rxj: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e200d546debfbb01f8e771fc9988ea9688ca35ab655eee8cfefd5596f261c5f4": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:48:33 +0000 UTC - event for pod-subpath-test-preprovisionedpv-7rxj: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0e32ade9daec1b4655c18c92786d02db2fdec89f6c8bfc73e4568d5e6e076717": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:48:47 +0000 UTC - event for pod-subpath-test-preprovisionedpv-7rxj: {kubelet i-0df245ff74f46fa48} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "034bfc42127bac146fabf8f2e6d7a88026ddeee6407df359e6d5f31f4ea5b239": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 25 06:51:53.558: INFO: At 2023-01-25 06:51:53 +0000 UTC - event for hostexec-i-0df245ff74f46fa48-xd5sx: {kubelet i-0df245ff74f46fa48} Killing: Stopping container agnhost-container
Jan 25 06:51:53.590: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 25 06:51:53.590: INFO:
Jan 25 06:51:53.622: INFO:
Logging node info for node i-071cd767a01ed5a98
Jan 25 06:51:53.654: INFO: Node Info: &Node{ObjectMeta:{i-071cd767a01ed5a98 e176a57e-e197-4621-9ff9-1551f1909075 46278 0 2023-01-25 06:25:20 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-071cd767a01ed5a98 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ca-central-1a topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.39.108 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-071cd767a01ed5a98"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-25 06:25:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-25 06:25:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:26:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:26:12 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-25 06:26:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-25 06:48:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-071cd767a01ed5a98,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3862937600 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3758080000 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:48:09 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:48:09 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:48:09 +0000 UTC,LastTransitionTime:2023-01-25 06:25:20 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:48:09 +0000 UTC,LastTransitionTime:2023-01-25 06:26:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.39.108,},NodeAddress{Type:ExternalIP,Address:3.97.7.38,},NodeAddress{Type:InternalDNS,Address:i-071cd767a01ed5a98.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-071cd767a01ed5a98.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-97-7-38.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dc6c1622927717fff709d069d68b2,SystemUUID:ec2dc6c1-6229-2771-7fff-709d069d68b2,BootID:034bcfe8-9462-416d-9a80-bb9dcb9fd35d,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.1],SizeBytes:135178704,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.1],SizeBytes:124995897,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:63920476,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.1],SizeBytes:57661752,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:44339025,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42841295,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4965790,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 25 06:51:53.654: INFO:
Logging kubelet events for node i-071cd767a01ed5a98
Jan 25 06:51:53.688: INFO:
Logging pods the kubelet thinks is on node i-071cd767a01ed5a98
Jan 25 06:51:53.730: INFO: etcd-manager-events-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (10+1 container statuses recorded)
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Container etcd-manager ready: true, restart count 0
Jan 25 06:51:53.730: INFO: etcd-manager-main-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (10+1 container statuses recorded)
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Container etcd-manager ready: true, restart count 0
Jan 25 06:51:53.730: INFO: kube-apiserver-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+2 container statuses recorded)
Jan 25 06:51:53.730: INFO: Container healthcheck ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Container kube-apiserver ready: true, restart count 3
Jan 25 06:51:53.730: INFO: kube-controller-manager-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:53.730: INFO: Container kube-controller-manager ready: true, restart count 4
Jan 25 06:51:53.730: INFO: kube-scheduler-i-071cd767a01ed5a98 started at 2023-01-25 06:23:34 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:53.730: INFO: Container kube-scheduler ready: true, restart count 0
Jan 25 06:51:53.730: INFO: cilium-nqkjx started at 2023-01-25 06:26:00 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:51:53.730: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:51:53.730: INFO: dns-controller-69987775c6-7nx57 started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:53.730: INFO: Container dns-controller ready: true, restart count 0
Jan 25 06:51:53.730: INFO: aws-cloud-controller-manager-6kpfv started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:53.730: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 25 06:51:53.730: INFO: kops-controller-qfzp7 started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:53.730: INFO: Container kops-controller ready: true, restart count 0
Jan 25 06:51:53.730: INFO: ebs-csi-controller-5bd98b456f-cdjzf started at 2023-01-25 06:26:00 +0000 UTC (0+5 container statuses recorded)
Jan 25 06:51:53.730: INFO: Container csi-attacher ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Container csi-resizer ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:51:53.730: INFO: ebs-csi-node-7t2hw started at 2023-01-25 06:26:00 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:51:53.730: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:51:53.730: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:51:53.730: INFO: cilium-operator-5dd44dc49f-jhvqd started at 2023-01-25 06:26:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:53.730: INFO: Container cilium-operator ready: true, restart count 0
Jan 25 06:51:53.902: INFO:
Latency metrics for node i-071cd767a01ed5a98
Jan 25 06:51:53.902: INFO:
Logging node info for node i-0dbcb4dc9e1f5e62b
Jan 25 06:51:53.934: INFO: Node Info: &Node{ObjectMeta:{i-0dbcb4dc9e1f5e62b f1ad0cae-2ed9-41a4-8be5-9884ae32adaa 48848 0 2023-01-25 06:27:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0dbcb4dc9e1f5e62b kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0dbcb4dc9e1f5e62b topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.63.11 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0dbcb4dc9e1f5e62b"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-25 06:46:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:51:35 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:51:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0dbcb4dc9e1f5e62b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:51:35 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:51:35 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:51:35 +0000 UTC,LastTransitionTime:2023-01-25 06:27:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:51:35 +0000 UTC,LastTransitionTime:2023-01-25 06:27:38 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.63.11,},NodeAddress{Type:ExternalIP,Address:3.96.138.144,},NodeAddress{Type:InternalDNS,Address:i-0dbcb4dc9e1f5e62b.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0dbcb4dc9e1f5e62b.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-138-144.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2cc8535b4596aa012ef7857efd2302,SystemUUID:ec2cc853-5b45-96aa-012e-f7857efd2302,BootID:1d85d5b3-ad1c-462b-98e2-f01921a4e990,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-089a5ec2b8005e4f1 kubernetes.io/csi/ebs.csi.aws.com^vol-0e695773ecdd510dc],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-089a5ec2b8005e4f1,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0e695773ecdd510dc,DevicePath:,},},Config:nil,},}
Jan 25 06:51:53.935: INFO:
Logging kubelet events for node i-0dbcb4dc9e1f5e62b
Jan 25 06:51:53.969: INFO:
Logging pods the kubelet thinks is on node i-0dbcb4dc9e1f5e62b
Jan 25 06:51:54.015: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-psvwb started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.015: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qmlhj started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.015: INFO: hostexec-i-0dbcb4dc9e1f5e62b-97cc8 started at 2023-01-25 06:50:00 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container agnhost-container ready: true, restart count 0
Jan 25 06:51:54.015: INFO: webserver-deployment-7f5969cbc7-pzv6w started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container httpd ready: false, restart count 0
Jan 25 06:51:54.015: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xqz46 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.015: INFO: pod-subpath-test-dynamicpv-gwtk started at 2023-01-25 06:51:51 +0000 UTC (2+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Init container init-volume-dynamicpv-gwtk ready: false, restart count 0
Jan 25 06:51:54.015: INFO: Init container test-init-volume-dynamicpv-gwtk ready: false, restart count 0
Jan 25 06:51:54.015: INFO: Container test-container-subpath-dynamicpv-gwtk ready: false, restart count 0
Jan 25 06:51:54.015: INFO: ebs-csi-node-bn5vq started at 2023-01-25 06:27:26 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:51:54.015: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:51:54.015: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:51:54.015: INFO: busybox-1b7825ea-2c0f-4fcb-b7f9-c97bd78ff608 started at 2023-01-25 06:50:33 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container busybox ready: false, restart count 0
Jan 25 06:51:54.015: INFO: webserver-deployment-7f5969cbc7-ctzws started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container httpd ready: false, restart count 0
Jan 25 06:51:54.015: INFO: inline-volume-tester-fs6qc started at 2023-01-25 06:51:24 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container csi-volume-tester ready: true, restart count 0
Jan 25 06:51:54.015: INFO: pod-configmaps-4094b428-7383-4e61-97b6-bae4d8f7a209 started at 2023-01-25 06:51:49 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container agnhost-container ready: false, restart count 0
Jan 25 06:51:54.015: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-gvw2h started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.015: INFO: cilium-wptjp started at 2023-01-25 06:27:26 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:51:54.015: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:51:54.015: INFO: local-injector started at 2023-01-25 06:50:06 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container local-injector ready: false, restart count 0
Jan 25 06:51:54.015: INFO: pod-d973dfcb-fb6f-420f-9eb8-855583292025 started at 2023-01-25 06:51:32 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container test-container ready: false, restart count 0
Jan 25 06:51:54.015: INFO: webserver-deployment-7f5969cbc7-fqq2n started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container httpd ready: true, restart count 0
Jan 25 06:51:54.015: INFO: csi-mockplugin-attacher-0 started at 2023-01-25 06:50:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container csi-attacher ready: false, restart count 0
Jan 25 06:51:54.015: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jjlq8 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.015: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rt5t5 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.015: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-8h8sv started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.015: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5kz2h started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.015: INFO: csi-mockplugin-0 started at 2023-01-25 06:50:15 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container csi-provisioner ready: false, restart count 0
Jan 25 06:51:54.015: INFO: Container driver-registrar ready: false, restart count 0
Jan 25 06:51:54.015: INFO: Container mock ready: false, restart count 0
Jan 25 06:51:54.015: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-4wfp7 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.015: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-dz9mt started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.015: INFO: csi-mockplugin-resizer-0 started at 2023-01-25 06:50:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.015: INFO: Container csi-resizer ready: false, restart count 0
Jan 25 06:51:54.289: INFO:
Latency metrics for node i-0dbcb4dc9e1f5e62b
Jan 25 06:51:54.289: INFO:
Logging node info for node i-0df245ff74f46fa48
Jan 25 06:51:54.320: INFO: Node Info: &Node{ObjectMeta:{i-0df245ff74f46fa48 3893284f-3580-49f6-b7a1-2ddba9ae06ec 48407 0 2023-01-25 06:27:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0df245ff74f46fa48 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0df245ff74f46fa48 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.41.87 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-6879":"i-0df245ff74f46fa48","ebs.csi.aws.com":"i-0df245ff74f46fa48"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kops-controller Update v1 2023-01-25 06:27:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {e2e.test Update v1 2023-01-25 06:46:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:50:23 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:50:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0df245ff74f46fa48,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:50:26 +0000 UTC,LastTransitionTime:2023-01-25 06:27:40 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.41.87,},NodeAddress{Type:ExternalIP,Address:3.96.184.142,},NodeAddress{Type:InternalDNS,Address:i-0df245ff74f46fa48.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0df245ff74f46fa48.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-184-142.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2129fadd2b2b52f98b14d7c0337fff,SystemUUID:ec2129fa-dd2b-2b52-f98b-14d7c0337fff,BootID:6b9e799a-3b2a-44dc-8a4e-1a86f3b5230b,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-020105115fe204770],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-020105115fe204770,DevicePath:,},},Config:nil,},}
Jan 25 06:51:54.321: INFO:
Logging kubelet events for node i-0df245ff74f46fa48
Jan 25 06:51:54.355: INFO:
Logging pods the kubelet thinks is on node i-0df245ff74f46fa48
Jan 25 06:51:54.395: INFO: ebs-csi-node-cfszw started at 2023-01-25 06:27:28 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:51:54.395: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:51:54.395: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:51:54.395: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x5hz9 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.395: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sxzm4 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.395: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bl4p9 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.395: INFO: pfpod started at 2023-01-25 06:49:34 +0000 UTC (0+2 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container portforwardtester ready: false, restart count 0
Jan 25 06:51:54.395: INFO: Container readiness ready: false, restart count 0
Jan 25 06:51:54.395: INFO: csi-mockplugin-0 started at 2023-01-25 06:46:38 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:51:54.395: INFO: Container driver-registrar ready: true, restart count 0
Jan 25 06:51:54.395: INFO: Container mock ready: true, restart count 0
Jan 25 06:51:54.395: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-569r5 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.395: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-plznd started at 2023-01-25 06:46:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.395: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ggrjc started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.395: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2mwzt started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.395: INFO: webserver-deployment-7f5969cbc7-q9lv4 started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container httpd ready: false, restart count 0
Jan 25 06:51:54.395: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hd52p started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.395: INFO: webserver-deployment-7f5969cbc7-hrp6g started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container httpd ready: false, restart count 0
Jan 25 06:51:54.395: INFO: webserver-deployment-7f5969cbc7-r4gfp started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container httpd ready: true, restart count 0
Jan 25 06:51:54.395: INFO: cilium-4kl5t started at 2023-01-25 06:27:28 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:51:54.395: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:51:54.395: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:51:54.395: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rzvq9 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.395: INFO: pod-b2d5aa59-ddb2-497b-b629-98cd25649a85 started at 2023-01-25 06:50:17 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container write-pod ready: false, restart count 0
Jan 25 06:51:54.395: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-szf5f started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.395: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.618: INFO:
Latency metrics for node i-0df245ff74f46fa48
Jan 25 06:51:54.618: INFO:
Logging node info for node i-0e8434bc9e1236fd5
Jan 25 06:51:54.650: INFO: Node Info: &Node{ObjectMeta:{i-0e8434bc9e1236fd5 48f670c2-fe1c-46d3-b197-464836dab89d 47890 0 2023-01-25 06:27:23 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0e8434bc9e1236fd5 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0e8434bc9e1236fd5 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.54.30 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-5972":"i-0e8434bc9e1236fd5","ebs.csi.aws.com":"i-0e8434bc9e1236fd5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-25 06:27:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-25 06:46:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:49:53 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:50:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0e8434bc9e1236fd5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:23 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:50:02 +0000 UTC,LastTransitionTime:2023-01-25 06:27:35 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.54.30,},NodeAddress{Type:ExternalIP,Address:99.79.58.56,},NodeAddress{Type:InternalDNS,Address:i-0e8434bc9e1236fd5.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0e8434bc9e1236fd5.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-99-79-58-56.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2439e74c65fcfb2f8c6ece88db3944,SystemUUID:ec2439e7-4c65-fcfb-2f8c-6ece88db3944,BootID:568b3843-a4d7-46ef-b502-19f3e8310af2,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-5972^782f81a1-9c7c-11ed-97c4-deff35c7067a kubernetes.io/csi/ebs.csi.aws.com^vol-058ab77a5848fba8a kubernetes.io/csi/ebs.csi.aws.com^vol-06d3e55226f87a3e7],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-058ab77a5848fba8a,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-5972^782f81a1-9c7c-11ed-97c4-deff35c7067a,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0458d17653ca96edd,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-06d3e55226f87a3e7,DevicePath:,},},Config:nil,},}
Jan 25 06:51:54.650: INFO:
Logging kubelet events for node i-0e8434bc9e1236fd5
Jan 25 06:51:54.684: INFO:
Logging pods the kubelet thinks is on node i-0e8434bc9e1236fd5
Jan 25 06:51:54.725: INFO: csi-mockplugin-0 started at 2023-01-25 06:49:32 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container csi-provisioner ready: false, restart count 0
Jan 25 06:51:54.726: INFO: Container driver-registrar ready: false, restart count 0
Jan 25 06:51:54.726: INFO: Container mock ready: false, restart count 0
Jan 25 06:51:54.726: INFO: hostexec-i-0e8434bc9e1236fd5-274ww started at 2023-01-25 06:50:47 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container agnhost-container ready: true, restart count 0
Jan 25 06:51:54.726: INFO: pod-9c25960d-410f-429b-b861-ab98bb2eb98b started at 2023-01-25 06:50:50 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container write-pod ready: false, restart count 0
Jan 25 06:51:54.726: INFO: webserver-deployment-7f5969cbc7-wwshs started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container httpd ready: false, restart count 0
Jan 25 06:51:54.726: INFO: ebs-csi-node-4d5tv started at 2023-01-25 06:27:24 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:51:54.726: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:51:54.726: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:51:54.726: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qqns6 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.726: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-jspbp started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.726: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-lw7tl started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.726: INFO: coredns-autoscaler-7cb5c5b969-mndhj started at 2023-01-25 06:27:35 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container autoscaler ready: true, restart count 0
Jan 25 06:51:54.726: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-tczwz started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.726: INFO: csi-mockplugin-attacher-0 started at 2023-01-25 06:49:32 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container csi-attacher ready: false, restart count 0
Jan 25 06:51:54.726: INFO: csi-hostpathplugin-0 started at 2023-01-25 06:49:50 +0000 UTC (0+7 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container csi-attacher ready: true, restart count 0
Jan 25 06:51:54.726: INFO: Container csi-provisioner ready: true, restart count 0
Jan 25 06:51:54.726: INFO: Container csi-resizer ready: true, restart count 0
Jan 25 06:51:54.726: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 25 06:51:54.726: INFO: Container hostpath ready: true, restart count 0
Jan 25 06:51:54.726: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:51:54.726: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:51:54.726: INFO: webserver-deployment-7f5969cbc7-t7ck4 started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container httpd ready: false, restart count 0
Jan 25 06:51:54.726: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-sr5k2 started at 2023-01-25 06:46:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.726: INFO: pod-c185a4fa-326c-4f33-957a-bddf24584c62 started at 2023-01-25 06:47:47 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container write-pod ready: false, restart count 0
Jan 25 06:51:54.726: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-rbghp started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.726: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hzhwc started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.726: INFO: pod-subpath-test-dynamicpv-mf6j started at 2023-01-25 06:49:53 +0000 UTC (0+2 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container test-container-subpath-dynamicpv-mf6j ready: false, restart count 0
Jan 25 06:51:54.726: INFO: Container test-container-volume-dynamicpv-mf6j ready: false, restart count 0
Jan 25 06:51:54.726: INFO: cilium-xkk5h started at 2023-01-25 06:27:24 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:51:54.726: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:51:54.726: INFO: inline-volume-tester-2shhw started at 2023-01-25 06:47:57 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 25 06:51:54.726: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-2kf7l started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.726: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-xrncj started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: false, restart count 0
Jan 25 06:51:54.726: INFO: pod-subpath-test-preprovisionedpv-vfbk started at 2023-01-25 06:51:06 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Init container init-volume-preprovisionedpv-vfbk ready: false, restart count 0
Jan 25 06:51:54.726: INFO: Container test-container-subpath-preprovisionedpv-vfbk ready: false, restart count 0
Jan 25 06:51:54.726: INFO: coredns-559769c974-ggjhw started at 2023-01-25 06:27:35 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container coredns ready: true, restart count 0
Jan 25 06:51:54.726: INFO: hostexec-i-0e8434bc9e1236fd5-9ql96 started at 2023-01-25 06:50:49 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container agnhost-container ready: true, restart count 0
Jan 25 06:51:54.726: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-5mlr8 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:54.726: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:54.941: INFO:
Latency metrics for node i-0e8434bc9e1236fd5
Jan 25 06:51:54.941: INFO:
Logging node info for node i-0ebbe4bd7ec58d213
Jan 25 06:51:54.973: INFO: Node Info: &Node{ObjectMeta:{i-0ebbe4bd7ec58d213 f69d5727-b51d-4595-ab9a-fe57b5c9c863 47765 0 2023-01-25 06:27:26 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0ebbe4bd7ec58d213 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0ebbe4bd7ec58d213 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.215 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0ebbe4bd7ec58d213"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-25 06:27:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-25 06:46:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-25 06:49:53 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-25 06:49:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0ebbe4bd7ec58d213,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051685376 0} {<nil>} 3956724Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946827776 0} {<nil>} 3854324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-25 06:49:56 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-25 06:49:56 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-25 06:49:56 +0000 UTC,LastTransitionTime:2023-01-25 06:27:26 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-25 06:49:56 +0000 UTC,LastTransitionTime:2023-01-25 06:27:38 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.215,},NodeAddress{Type:ExternalIP,Address:3.96.126.35,},NodeAddress{Type:InternalDNS,Address:i-0ebbe4bd7ec58d213.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ebbe4bd7ec58d213.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-126-35.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2da1db9b923c887fdbe1dd4557a0d5,SystemUUID:ec2da1db-9b92-3c88-7fdb-e1dd4557a0d5,BootID:10184fa3-2e15-48fe-9718-382b9d68823e,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-03294c638eea79454 kubernetes.io/csi/ebs.csi.aws.com^vol-0d005d0140f2dadfb],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-03294c638eea79454,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d005d0140f2dadfb,DevicePath:,},},Config:nil,},}
Jan 25 06:51:54.973: INFO:
Logging kubelet events for node i-0ebbe4bd7ec58d213
Jan 25 06:51:55.010: INFO:
Logging pods the kubelet thinks is on node i-0ebbe4bd7ec58d213
Jan 25 06:51:55.055: INFO: pod-disruption-failure-ignore-1-f77bm started at 2023-01-25 06:46:49 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container c ready: false, restart count 0
Jan 25 06:51:55.055: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-k9wft started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:55.055: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-qs6zz started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:55.055: INFO: pod-disruption-failure-ignore-0-l6qmf started at 2023-01-25 06:46:49 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container c ready: true, restart count 0
Jan 25 06:51:55.055: INFO: inline-volume-tester-bns44 started at 2023-01-25 06:49:50 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 25 06:51:55.055: INFO: webserver-deployment-7f5969cbc7-wch65 started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container httpd ready: false, restart count 0
Jan 25 06:51:55.055: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-npbz4 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:55.055: INFO: cilium-pkfh6 started at 2023-01-25 06:27:27 +0000 UTC (1+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 25 06:51:55.055: INFO: Container cilium-agent ready: true, restart count 0
Jan 25 06:51:55.055: INFO: coredns-559769c974-v87jf started at 2023-01-25 06:27:57 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container coredns ready: true, restart count 0
Jan 25 06:51:55.055: INFO: pod-7dc5023a-bc51-44e0-a5b1-2ac8cbb4d9a7 started at 2023-01-25 06:51:20 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container write-pod ready: false, restart count 0
Jan 25 06:51:55.055: INFO: pod-disruption-failure-ignore-1-vmrs8 started at 2023-01-25 06:47:22 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container c ready: false, restart count 0
Jan 25 06:51:55.055: INFO: csi-hostpathplugin-0 started at 2023-01-25 06:49:47 +0000 UTC (0+7 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container csi-attacher ready: false, restart count 0
Jan 25 06:51:55.055: INFO: Container csi-provisioner ready: false, restart count 0
Jan 25 06:51:55.055: INFO: Container csi-resizer ready: false, restart count 0
Jan 25 06:51:55.055: INFO: Container csi-snapshotter ready: false, restart count 0
Jan 25 06:51:55.055: INFO: Container hostpath ready: false, restart count 0
Jan 25 06:51:55.055: INFO: Container liveness-probe ready: false, restart count 0
Jan 25 06:51:55.055: INFO: Container node-driver-registrar ready: false, restart count 0
Jan 25 06:51:55.055: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-7zg9l started at 2023-01-25 06:46:16 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:55.055: INFO: webserver-deployment-7f5969cbc7-wg5sj started at 2023-01-25 06:51:03 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container httpd ready: false, restart count 0
Jan 25 06:51:55.055: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-wm7lb started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:55.055: INFO: pod-77e2c30c-da83-47c3-846b-f0c680a63e2d started at 2023-01-25 06:49:59 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container write-pod ready: false, restart count 0
Jan 25 06:51:55.055: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-ft4g2 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:55.055: INFO: hostexec-i-0ebbe4bd7ec58d213-9jht8 started at 2023-01-25 06:49:56 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container agnhost-container ready: true, restart count 0
Jan 25 06:51:55.055: INFO: external-injector started at 2023-01-25 06:49:33 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container external-injector ready: false, restart count 0
Jan 25 06:51:55.055: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-x4pht started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:55.055: INFO: hostexec-i-0ebbe4bd7ec58d213-brbmd started at 2023-01-25 06:51:14 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container agnhost-container ready: true, restart count 0
Jan 25 06:51:55.055: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-hqhjx started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:55.055: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-csrp6 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:55.055: INFO: cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8-bpq52 started at 2023-01-25 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container cleanup40-0bfa8122-2f6f-457e-adba-d29cee9558d8 ready: true, restart count 0
Jan 25 06:51:55.055: INFO: ebs-csi-node-9ngsq started at 2023-01-25 06:27:27 +0000 UTC (0+3 container statuses recorded)
Jan 25 06:51:55.055: INFO: Container ebs-plugin ready: true, restart count 0
Jan 25 06:51:55.055: INFO: Container liveness-probe ready: true, restart count 0
Jan 25 06:51:55.055: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 25 06:51:55.371: INFO:
Latency metrics for node i-0ebbe4bd7ec58d213
[DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "provisioning-2863" for this suite. �[38;5;243m01/25/23 06:51:55.371�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
exit status 255
from junit_runner.xml
Filter through log files | View test history on testgrid
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand Verify if offline PVC expansion works
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand should resize volume when PVC is edited while pod is using it
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (block volmode)] volume-expand should not allow expansion of pvcs without AllowVolumeExpansion property
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (block volmode)] volumes should store data
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand Verify if offline PVC expansion works
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand should resize volume when PVC is edited while pod is using it
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with different fsgroup applied to the volume contents
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with same fsgroup applied to the volume contents
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with different fsgroup applied to the volume contents
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with same fsgroup skips ownership changes to the volume contents
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with mount options
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] subPath should be able to unmount after the subpath directory is deleted [LinuxOnly]
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directories when readOnly specified in the volumeSource
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directory
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing single file [LinuxOnly]
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] subPath should support file as subpath [LinuxOnly]
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] subPath should support non-existent path
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly directory specified in the volumeMount
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly file specified in the volumeMount [LinuxOnly]
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] volume-expand should not allow expansion of pvcs without AllowVolumeExpansion property
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (default fs)] volumes should allow exec of files on the volume
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (delayed binding)] topology should provision a volume and schedule a pod with AllowedTopologies
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (ext4)] volumes should allow exec of files on the volume
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (ext4)] volumes should store data
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits should verify that all csinodes have volume limits
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode should not mount / map unused volumes in a pod [LinuxOnly]
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Dynamic PV (immediate binding)] topology should provision a volume and schedule a pod with AllowedTopologies
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Generic Ephemeral-volume (block volmode) (late-binding)] ephemeral should create read/write inline ephemeral volume
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Generic Ephemeral-volume (block volmode) (late-binding)] ephemeral should support expansion of pvcs created for ephemeral pvcs
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Generic Ephemeral-volume (block volmode) (late-binding)] ephemeral should support multiple inline ephemeral volumes
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Generic Ephemeral-volume (block volmode) (late-binding)] ephemeral should support two pods which have the same volume definition
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read-only inline ephemeral volume
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read/write inline ephemeral volume
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support expansion of pvcs created for ephemeral pvcs
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which have the same volume definition
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read-only inline ephemeral volume
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read/write inline ephemeral volume
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support expansion of pvcs created for ephemeral pvcs
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support multiple inline ephemeral volumes
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support two pods which have the same volume definition
Kubernetes e2e suite [It] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Generic Ephemeral-volume (default fs)] volumeLimits should verify that all csinodes have volume limits
Kubernetes e2e suite [It] [sig-api-machinery] API priority and fairness should ensure that requests can be classified by adding FlowSchema and PriorityLevelConfiguration
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] listing mutating webhooks should work [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] listing validating webhooks should work [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] patching/updating a mutating webhook should work [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] patching/updating a validating webhook should work [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should be able to deny attaching pod [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should be able to deny custom resource creation, update and deletion [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should be able to deny pod and configmap creation [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should deny crd creation [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should honor timeout [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should include webhook resources in discovery documents [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate configmap [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate custom resource [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate custom resource with different stored version [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate custom resource with pruning [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate pod and apply defaults after mutation [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should not be able to mutate or prevent deletion of webhook configuration objects [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should unconditionally reject operations on fail closed webhook [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] Aggregator Should be able to support the 1.17 Sample API Server using the current Aggregator [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceConversionWebhook [Privileged:ClusterAdmin] should be able to convert a non homogeneous list of CRs [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceConversionWebhook [Privileged:ClusterAdmin] should be able to convert from CR v1 to CR v2 [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceDefinition Watch [Privileged:ClusterAdmin] CustomResourceDefinition Watch watch on custom resource definition objects [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] Simple CustomResourceDefinition creating/deleting custom resource definition objects works [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] Simple CustomResourceDefinition getting/updating/patching custom resource definition status sub-resource works [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] Simple CustomResourceDefinition listing custom resource definition objects works [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] custom resource defaulting for requests and from storage works [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] should include custom resource definition resources in discovery documents [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] removes definition from spec when one version gets changed to not be served [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] updates the published spec when one version gets renamed [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] works for CRD preserving unknown fields at the schema root [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] works for CRD preserving unknown fields in an embedded object [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] works for CRD with validation schema [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] works for CRD without validation schema [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] works for multiple CRDs of different groups [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] works for multiple CRDs of same group and version but different kinds [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] works for multiple CRDs of same group but different versions [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] MUST NOT fail validation for create of a custom resource that satisfies the x-kubernetes-validations rules
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] MUST fail create of a custom resource definition that contains a x-kubernetes-validations rule that refers to a property that do not exist
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] MUST fail create of a custom resource definition that contains an x-kubernetes-validations rule that contains a syntax error
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] MUST fail create of a custom resource definition that contains an x-kubernetes-validations rule that exceeds the estimated cost limit
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] MUST fail create of a custom resource that exceeds the runtime cost limit for x-kubernetes-validations rule execution
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] MUST fail update of a custom resource that does not satisfy a x-kubernetes-validations transition rule
Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] MUST fail validation for create of a custom resource that does not satisfy the x-kubernetes-validations rules
Kubernetes e2e suite [It] [sig-api-machinery] Discovery Custom resource should have storage version hash
Kubernetes e2e suite [It] [sig-api-machinery] Discovery should accurately determine present and missing resources
Kubernetes e2e suite [It] [sig-api-machinery] Discovery should validate PreferredVersion for each APIGroup [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] Garbage collector should delete RS created by deployment when not orphaning [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] Garbage collector should delete jobs and pods created by cronjob
Kubernetes e2e suite [It] [sig-api-machinery] Garbage collector should delete pods created by rc when not orphaning [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] Garbage collector should keep the rc around until all its pods are deleted if the deleteOptions says so [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] Garbage collector should not be blocked by dependency circle [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] Garbage collector should not delete dependents that have both valid owner and owner that's waiting for dependents to be deleted [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] Garbage collector should orphan RS created by deployment when deleteOptions.PropagationPolicy is Orphan [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] Garbage collector should orphan pods created by rc if delete options say so [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] Garbage collector should orphan pods created by rc if deleteOptions.OrphanDependents is nil
Kubernetes e2e suite [It] [sig-api-machinery] Garbage collector should support cascading deletion of custom resources
Kubernetes e2e suite [It] [sig-api-machinery] Garbage collector should support orphan deletion of custom resources
Kubernetes e2e suite [It] [sig-api-machinery] Generated clientset should create pods, set the deletionTimestamp and deletionGracePeriodSeconds of the pod
Kubernetes e2e suite [It] [sig-api-machinery] Generated clientset should create v1 cronJobs, delete cronJobs, watch cronJobs
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should apply changes to a resourcequota status [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should be able to update and delete ResourceQuota. [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a configMap. [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a custom resource.
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a persistent volume claim
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a persistent volume claim with a storage class
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a pod. [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a replica set. [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a replication controller. [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a secret. [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a service. [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should create a ResourceQuota and ensure its status is promptly calculated. [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should manage the lifecycle of a ResourceQuota [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should verify ResourceQuota with best effort scope. [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should verify ResourceQuota with cross namespace pod affinity scope using scope-selectors.
Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should verify ResourceQuota with terminating scopes. [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] Server request timeout default timeout should be used if the specified timeout in the request URL is 0s
Kubernetes e2e suite [It] [sig-api-machinery] Server request timeout should return HTTP status code 400 if the user specifies an invalid timeout in the request URL
Kubernetes e2e suite [It] [sig-api-machinery] Server request timeout the request should be served with a default timeout if the specified timeout in the request URL exceeds maximum allowed
Kubernetes e2e suite [It] [sig-api-machinery] ServerSideApply should create an applied object if it does not already exist
Kubernetes e2e suite [It] [sig-api-machinery] ServerSideApply should give up ownership of a field if forced applied by a controller
Kubernetes e2e suite [It] [sig-api-machinery] ServerSideApply should ignore conflict errors if force apply is used
Kubernetes e2e suite [It] [sig-api-machinery] ServerSideApply should not remove a field if an owner unsets the field but other managers still have ownership of the field
Kubernetes e2e suite [It] [sig-api-machinery] ServerSideApply should remove a field if it is owned but removed in the apply request
Kubernetes e2e suite [It] [sig-api-machinery] ServerSideApply should work for CRDs
Kubernetes e2e suite [It] [sig-api-machinery] ServerSideApply should work for subresources
Kubernetes e2e suite [It] [sig-api-machinery] Servers with support for API chunking should return chunks of results for list calls
Kubernetes e2e suite [It] [sig-api-machinery] Servers with support for Table transformation should return a 406 for a backend which does not implement metadata [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] Servers with support for Table transformation should return chunks of table results for list calls
Kubernetes e2e suite [It] [sig-api-machinery] Servers with support for Table transformation should return generic metadata details across all namespaces for nodes
Kubernetes e2e suite [It] [sig-api-machinery] Servers with support for Table transformation should return pod details
Kubernetes e2e suite [It] [sig-api-machinery] Watchers should be able to restart watching from the last resource version observed by the previous watch [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] Watchers should be able to start watching from a specific resource version [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] Watchers should observe add, update, and delete watch notifications on configmaps [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] Watchers should observe an object deletion if it stops meeting the requirements of the selector [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] Watchers should receive events on concurrent watches in same order [Conformance]
Kubernetes e2e suite [It] [sig-api-machinery] client-go should negotiate watch and report errors with accept "application/json"
Kubernetes e2e suite [It] [sig-api-machinery] client-go should negotiate watch and report errors with accept "application/json,application/vnd.kubernetes.protobuf"
Kubernetes e2e suite [It] [sig-api-machinery] client-go should negotiate watch and report errors with accept "application/vnd.kubernetes.protobuf"
Kubernetes e2e suite [It] [sig-api-machinery] client-go should negotiate watch and report errors with accept "application/vnd.kubernetes.protobuf,application/json"
Kubernetes e2e suite [It] [sig-api-machinery] health handlers should contain necessary checks
Kubernetes e2e suite [It] [sig-api-machinery] server version should find the server version [Conformance]
Kubernetes e2e suite [It] [sig-apps] CronJob should be able to schedule after more than 100 missed schedule
Kubernetes e2e suite [It] [sig-apps] CronJob should delete failed finished jobs with limit of one job
Kubernetes e2e suite [It] [sig-apps] CronJob should delete successful finished jobs with limit of one successful job
Kubernetes e2e suite [It] [sig-apps] CronJob should not emit unexpected warnings
Kubernetes e2e suite [It] [sig-apps] CronJob should remove from active list jobs that have been deleted
Kubernetes e2e suite [It] [sig-apps] CronJob should replace jobs when ReplaceConcurrent [Conformance]
Kubernetes e2e suite [It] [sig-apps] CronJob should schedule multiple jobs concurrently [Conformance]
Kubernetes e2e suite [It] [sig-apps] CronJob should support CronJob API operations [Conformance]
Kubernetes e2e suite [It] [sig-apps] CronJob should support timezone
Kubernetes e2e suite [It] [sig-apps] Deployment Deployment should have a working scale subresource [Conformance]
Kubernetes e2e suite [It] [sig-apps] Deployment RecreateDeployment should delete old pods and create new ones [Conformance]
Kubernetes e2e suite [It] [sig-apps] Deployment RollingUpdateDeployment should delete old pods and create new ones [Conformance]
Kubernetes e2e suite [It] [sig-apps] Deployment deployment reaping should cascade to its replica sets and pods
Kubernetes e2e suite [It] [sig-apps] Deployment deployment should delete old replica sets [Conformance]
Kubernetes e2e suite [It] [sig-apps] Deployment deployment should support proportional scaling [Conformance]
Kubernetes e2e suite [It] [sig-apps] Deployment deployment should support rollover [Conformance]
Kubernetes e2e suite [It] [sig-apps] Deployment iterative rollouts should eventually progress
Kubernetes e2e suite [It] [sig-apps] Deployment should not disrupt a cloud load-balancer's connectivity during rollout
Kubernetes e2e suite [It] [sig-apps] Deployment should run the lifecycle of a Deployment [Conformance]
Kubernetes e2e suite [It] [sig-apps] Deployment should validate Deployment Status endpoints [Conformance]
Kubernetes e2e suite [It] [sig-apps] Deployment test Deployment ReplicaSet orphaning and adoption regarding controllerRef
Kubernetes e2e suite [It] [sig-apps] DisruptionController Listing PodDisruptionBudgets for all namespaces should list and delete a collection of PodDisruptionBudgets [Conformance]
Kubernetes e2e suite [It] [sig-apps] DisruptionController evictions: enough pods, absolute => should allow an eviction
Kubernetes e2e suite [It] [sig-apps] DisruptionController evictions: enough pods, replicaSet, percentage => should allow an eviction
Kubernetes e2e suite [It] [sig-apps] DisruptionController evictions: maxUnavailable allow single eviction, percentage => should allow an eviction
Kubernetes e2e suite [It] [sig-apps] DisruptionController evictions: no PDB => should allow an eviction
Kubernetes e2e suite [It] [sig-apps] DisruptionController evictions: too few pods, absolute => should not allow an eviction
Kubernetes e2e suite [It] [sig-apps] DisruptionController should block an eviction until the PDB is updated to allow it [Conformance]
Kubernetes e2e suite [It] [sig-apps] DisruptionController should create a PodDisruptionBudget [Conformance]
Kubernetes e2e suite [It] [sig-apps] DisruptionController should observe PodDisruptionBudget status updated [Conformance]
Kubernetes e2e suite [It] [sig-apps] DisruptionController should observe that the PodDisruptionBudget status is not updated for unmanaged pods
Kubernetes e2e suite [It] [sig-apps] DisruptionController should update/patch PodDisruptionBudget status [Conformance]
Kubernetes e2e suite [It] [sig-apps] Job Using a pod failure policy to not count some failures towards the backoffLimit Ignore DisruptionTarget condition
Kubernetes e2e suite [It] [sig-apps] Job Using a pod failure policy to not count some failures towards the backoffLimit Ignore exit code 137
Kubernetes e2e suite [It] [sig-apps] Job should adopt matching orphans and release non-matching pods [Conformance]
Kubernetes e2e suite [It] [sig-apps] Job should allow to use the pod failure policy on exit code to fail the job early
Kubernetes e2e suite [It] [sig-apps] Job should allow to use the pod failure policy to not count the failure towards the backoffLimit
Kubernetes e2e suite [It] [sig-apps] Job should apply changes to a job status [Conformance]
Kubernetes e2e suite [It] [sig-apps] Job should create pods for an Indexed job with completion indexes and specified hostname [Conformance]
Kubernetes e2e suite [It] [sig-apps] Job should delete a job [Conformance]
Kubernetes e2e suite [It] [sig-apps] Job should delete pods when suspended
Kubernetes e2e suite [It] [sig-apps] Job should fail to exceed backoffLimit
Kubernetes e2e suite [It] [sig-apps] Job should fail when exceeds active deadline
Kubernetes e2e suite [It] [sig-apps] Job should manage the lifecycle of a job [Conformance]
Kubernetes e2e suite [It] [sig-apps] Job should not create pods when created in suspend state
Kubernetes e2e suite [It] [sig-apps] Job should remove pods when job is deleted
Kubernetes e2e suite [It] [sig-apps] Job should run a job to completion when tasks sometimes fail and are locally restarted [Conformance]
Kubernetes e2e suite [It] [sig-apps] Job should run a job to completion when tasks sometimes fail and are not locally restarted
Kubernetes e2e suite [It] [sig-apps] Job should run a job to completion when tasks succeed
Kubernetes e2e suite [It] [sig-apps] ReplicaSet Replace and Patch tests [Conformance]
Kubernetes e2e suite [It] [sig-apps] ReplicaSet Replicaset should have a working scale subresource [Conformance]
Kubernetes e2e suite [It] [sig-apps] ReplicaSet should adopt matching pods on creation and release no longer matching pods [Conformance]
Kubernetes e2e suite [It] [sig-apps] ReplicaSet should list and delete a collection of ReplicaSets [Conformance]
Kubernetes e2e suite [It] [sig-apps] ReplicaSet should serve a basic image on each replica with a public image [Conformance]
Kubernetes e2e suite [It] [sig-apps] ReplicaSet should surface a failure condition on a common issue like exceeded quota
Kubernetes e2e suite [It] [sig-apps] ReplicaSet should validate Replicaset Status endpoints [Conformance]
Kubernetes e2e suite [It] [sig-apps] ReplicationController should adopt matching pods on creation [Conformance]
Kubernetes e2e suite [It] [sig-apps] ReplicationController should get and update a ReplicationController scale [Conformance]
Kubernetes e2e suite [It] [sig-apps] ReplicationController should release no longer matching pods [Conformance]
Kubernetes e2e suite [It] [sig-apps] ReplicationController should serve a basic image on each replica with a public image [Conformance]
Kubernetes e2e suite [It] [sig-apps] ReplicationController should surface a failure condition on a common issue like exceeded quota [Conformance]
Kubernetes e2e suite [It] [sig-apps] ReplicationController should test the lifecycle of a ReplicationController [Conformance]
Kubernetes e2e suite [It] [sig-apps] StatefulSet AvailableReplicas should get updated accordingly when MinReadySeconds is enabled
Kubernetes e2e suite [It] [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Should recreate evicted statefulset [Conformance]
Kubernetes e2e suite [It] [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] should adopt matching orphans and release non-matching pods
Kubernetes e2e suite [It] [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] should have a working scale subresource [Conformance]
Kubernetes e2e suite [It] [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] should implement legacy replacement when the update strategy is OnDelete
Kubernetes e2e suite [It] [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] should list, patch and delete a collection of StatefulSets [Conformance]
Kubernetes e2e suite [It] [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] should not deadlock when a pod's predecessor fails
Kubernetes e2e suite [It] [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] should perform canary updates and phased rolling updates of template modifications [Conformance]
Kubernetes e2e suite [It] [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] should perform rolling updates and roll backs of template modifications [Conformance]
Kubernetes e2e suite [It] [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] should perform rolling updates and roll backs of template modifications with PVCs
Kubernetes e2e suite [It] [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] should provide basic identity
Kubernetes e2e suite [It] [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] should validate Statefulset Status endpoints [Conformance]
Kubernetes e2e suite [It] [sig-apps] StatefulSet MinReadySeconds should be honored when enabled
Kubernetes e2e suite [It] [sig-apps] TTLAfterFinished job should be deleted once it finishes after TTL seconds
Kubernetes e2e suite [It] [sig-architecture] Conformance Tests should have at least two untainted nodes [Conformance]
Kubernetes e2e suite [It] [sig-auth] Certificates API [Privileged:ClusterAdmin] should support CSR API operations [Conformance]
Kubernetes e2e suite [It] [sig-auth] Certificates API [Privileged:ClusterAdmin] should support building a client with a CSR
Kubernetes e2e suite [It] [sig-auth] ServiceAccounts ServiceAccountIssuerDiscovery should support OIDC discovery of service account issuer [Conformance]
Kubernetes e2e suite [It] [sig-auth] ServiceAccounts no secret-based service account token should be auto-generated
Kubernetes e2e suite [It] [sig-auth] ServiceAccounts should allow opting out of API token automount [Conformance]
Kubernetes e2e suite [It] [sig-auth] ServiceAccounts should guarantee kube-root-ca.crt exist in any namespace [Conformance]
Kubernetes e2e suite [It] [sig-auth] ServiceAccounts should mount an API token into pods [Conformance]
Kubernetes e2e suite [It] [sig-auth] ServiceAccounts should mount projected service account token [Conformance]
Kubernetes e2e suite [It] [sig-auth] ServiceAccounts should run through the lifecycle of a ServiceAccount [Conformance]
Kubernetes e2e suite [It] [sig-auth] ServiceAccounts should set ownership and permission when RunAsUser or FsGroup is present [LinuxOnly] [NodeFeature:FSGroup]
Kubernetes e2e suite [It] [sig-auth] ServiceAccounts should update a ServiceAccount [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl Port forwarding With a server listening on 0.0.0.0 that expects NO client request should support a client that connects, sends DATA, and disconnects
Kubernetes e2e suite [It] [sig-cli] Kubectl Port forwarding With a server listening on 0.0.0.0 that expects a client request should support a client that connects, sends DATA, and disconnects
Kubernetes e2e suite [It] [sig-cli] Kubectl Port forwarding With a server listening on 0.0.0.0 that expects a client request should support a client that connects, sends NO DATA, and disconnects
Kubernetes e2e suite [It] [sig-cli] Kubectl Port forwarding With a server listening on localhost should support forwarding over websockets
Kubernetes e2e suite [It] [sig-cli] Kubectl Port forwarding With a server listening on localhost that expects NO client request should support a client that connects, sends DATA, and disconnects
Kubernetes e2e suite [It] [sig-cli] Kubectl Port forwarding With a server listening on localhost that expects a client request should support a client that connects, sends DATA, and disconnects
Kubernetes e2e suite [It] [sig-cli] Kubectl Port forwarding With a server listening on localhost that expects a client request should support a client that connects, sends NO DATA, and disconnects
Kubernetes e2e suite [It] [sig-cli] Kubectl client Guestbook application should create and stop a working application [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl api-versions should check if v1 is in available api versions [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl apply apply set/view last-applied
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl apply should apply a new configuration to an existing RC
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl apply should reuse port when apply to an existing SVC
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl cluster-info dump should check if cluster-info dump succeeds
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl cluster-info should check if Kubernetes control plane services is included in cluster-info [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl copy should copy a file from a running Pod
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl create quota should create a quota with scopes
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl create quota should create a quota without scopes
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl create quota should reject quota with invalid scopes
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl describe should check if kubectl describe prints relevant information for cronjob
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl describe should check if kubectl describe prints relevant information for rc and pods [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl diff should check if kubectl diff finds a difference for Deployments [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl events should show event when pod is created
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl expose should create services for rc [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl get componentstatuses should get componentstatuses
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl label should update the label on a resource [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl logs should be able to retrieve and filter logs [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl patch should add annotations for pods in rc [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl replace should update a single-container pod's image [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl run pod should create a pod from an image when restart is Never [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl server-side dry-run should check if kubectl can dry-run update Pods [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl validation should create/apply a CR with unknown fields for CRD with no validation schema
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl validation should create/apply a valid CR for CRD with validation schema
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl validation should create/apply an invalid/valid CR with arbitrary-extra properties for CRD with partially-specified validation schema
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl validation should detect unknown metadata fields in both the root and embedded object of a CR
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl validation should detect unknown metadata fields of a typed object
Kubernetes e2e suite [It] [sig-cli] Kubectl client Kubectl version should check is all data is printed [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Proxy server should support --unix-socket=/path [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Proxy server should support proxy with --port 0 [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Simple pod should contain last line of the log
Kubernetes e2e suite [It] [sig-cli] Kubectl client Simple pod should handle in-cluster config
Kubernetes e2e suite [It] [sig-cli] Kubectl client Simple pod should return command exit codes execing into a container with a failing command
Kubernetes e2e suite [It] [sig-cli] Kubectl client Simple pod should return command exit codes execing into a container with a successful command
Kubernetes e2e suite [It] [sig-cli] Kubectl client Simple pod should return command exit codes running a failing command
Kubernetes e2e suite [It] [sig-cli] Kubectl client Simple pod should return command exit codes running a successful command
Kubernetes e2e suite [It] [sig-cli] Kubectl client Simple pod should support exec
Kubernetes e2e suite [It] [sig-cli] Kubectl client Simple pod should support exec through an HTTP proxy
Kubernetes e2e suite [It] [sig-cli] Kubectl client Simple pod should support exec through kubectl proxy
Kubernetes e2e suite [It] [sig-cli] Kubectl client Simple pod should support exec using resource/name
Kubernetes e2e suite [It] [sig-cli] Kubectl client Simple pod should support inline execution and attach
Kubernetes e2e suite [It] [sig-cli] Kubectl client Simple pod should support port-forward
Kubernetes e2e suite [It] [sig-cli] Kubectl client Update Demo should create and stop a replication controller [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client Update Demo should scale a replication controller [Conformance]
Kubernetes e2e suite [It] [sig-cli] Kubectl client kubectl wait should ignore not found error with --for=delete
Kubernetes e2e suite [It] [sig-instrumentation] Events API should delete a collection of events [Conformance]
Kubernetes e2e suite [It] [sig-instrumentation] Events API should ensure that an event can be fetched, patched, deleted, and listed [Conformance]
Kubernetes e2e suite [It] [sig-instrumentation] Events should delete a collection of events [Conformance]
Kubernetes e2e suite [It] [sig-instrumentation] Events should manage the lifecycle of an event [Conformance]
Kubernetes e2e suite [It] [sig-instrumentation] MetricsGrabber should grab all metrics from API server.
Kubernetes e2e suite [It] [sig-instrumentation] MetricsGrabber should grab all metrics from a ControllerManager.
Kubernetes e2e suite [It] [sig-instrumentation] MetricsGrabber should grab all metrics from a Kubelet.
Kubernetes e2e suite [It] [sig-instrumentation] MetricsGrabber should grab all metrics from a Scheduler.
Kubernetes e2e suite [It] [sig-network] CVE-2021-29923 IPv4 Service Type ClusterIP with leading zeros should work interpreted as decimal
Kubernetes e2e suite [It] [sig-network] Conntrack should be able to preserve UDP traffic when initial unready endpoints get ready
Kubernetes e2e suite [It] [sig-network] Conntrack should be able to preserve UDP traffic when server pod cycles for a ClusterIP service
Kubernetes e2e suite [It] [sig-network] Conntrack should be able to preserve UDP traffic when server pod cycles for a NodePort service
Kubernetes e2e suite [It] [sig-network] Conntrack should drop INVALID conntrack entries [Privileged]
Kubernetes e2e suite [It] [sig-network] DNS HostNetwork should resolve DNS of partial qualified names for services on hostNetwork pods with dnsPolicy: ClusterFirstWithHostNet [LinuxOnly]
Kubernetes e2e suite [It] [sig-network] DNS should provide /etc/hosts entries for the cluster [Conformance]
Kubernetes e2e suite [It] [sig-network] DNS should provide DNS for ExternalName services [Conformance]
Kubernetes e2e suite [It] [sig-network] DNS should provide DNS for pods for Hostname [Conformance]
Kubernetes e2e suite [It] [sig-network] DNS should provide DNS for pods for Subdomain [Conformance]
Kubernetes e2e suite [It] [sig-network] DNS should provide DNS for services [Conformance]
Kubernetes e2e suite [It] [sig-network] DNS should provide DNS for the cluster [Conformance]
Kubernetes e2e suite [It] [sig-network] DNS should resolve DNS of partial qualified names for services [LinuxOnly] [Conformance]
Kubernetes e2e suite [It] [sig-network] DNS should resolve DNS of partial qualified names for the cluster [LinuxOnly]
Kubernetes e2e suite [It] [sig-network] DNS should support configurable pod DNS nameservers [Conformance]
Kubernetes e2e suite [It] [sig-network] DNS should support configurable pod resolv.conf
Kubernetes e2e suite [It] [sig-network] DNS should work with the pod containing more than 6 DNS search paths and longer than 256 search list characters
Kubernetes e2e suite [It] [sig-network] EndpointSlice should create Endpoints and EndpointSlices for Pods matching a Service [Conformance]
Kubernetes e2e suite [It] [sig-network] EndpointSlice should create and delete Endpoints and EndpointSlices for a Service with a selector specified [Conformance]
Kubernetes e2e suite [It] [sig-network] EndpointSlice should have Endpoints and EndpointSlices pointing to API Server [Conformance]
Kubernetes e2e suite [It] [sig-network] EndpointSlice should support creating EndpointSlice API operations [Conformance]
Kubernetes e2e suite [It] [sig-network] EndpointSliceMirroring should mirror a custom Endpoints resource through create update and delete [Conformance]
Kubernetes e2e suite [It] [sig-network] Ingress API should support creating Ingress API operations [Conformance]
Kubernetes e2e suite [It] [sig-network] IngressClass API should support creating IngressClass API operations [Conformance]
Kubernetes e2e suite