e2e-kops-grid-cilium-eni-u2004-k26 #1611970796043898880
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 37 failed / 849 succeeded |
| Started | |
| Elapsed | 48m12s |
| Revision | master |
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(Always\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\svia\schgrp\sin\sfirst\spod\,\snew\spod\swith\sdifferent\sfsgroup\sapplied\sto\sthe\svolume\scontents$'
test/e2e/storage/testsuites/fsgroupchangepolicy.go:258
k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc001420f00, 0xc000f43e10, 0x0, {0xc004b5be10, 0x4}, {0xc004b5be14, 0x4})
test/e2e/storage/testsuites/fsgroupchangepolicy.go:258 +0x151
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3()
test/e2e/storage/testsuites/fsgroupchangepolicy.go:247 +0x556
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/08/23 06:46:09.322�[0m
Jan 8 06:46:09.322: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename fsgroupchangepolicy �[38;5;243m01/08/23 06:46:09.323�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/08/23 06:46:09.855�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/08/23 06:46:10.206�[0m
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
test/e2e/framework/metrics/init/init.go:31
[It] (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with different fsgroup applied to the volume contents
test/e2e/storage/testsuites/fsgroupchangepolicy.go:210
Jan 8 06:46:10.558: INFO: Creating resource for dynamic PV
Jan 8 06:46:10.558: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi}
�[1mSTEP:�[0m creating a StorageClass fsgroupchangepolicy-5020-e2e-scdnpl2 �[38;5;243m01/08/23 06:46:10.559�[0m
�[1mSTEP:�[0m creating a claim �[38;5;243m01/08/23 06:46:10.737�[0m
Jan 8 06:46:10.737: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
�[1mSTEP:�[0m Creating Pod in namespace fsgroupchangepolicy-5020 with fsgroup 1000 �[38;5;243m01/08/23 06:46:11.091�[0m
Jan 8 06:46:11.271: INFO: Waiting up to 5m0s for pod "pod-fd1b0f03-d676-45da-8d9a-f04db4320846" in namespace "fsgroupchangepolicy-5020" to be "running"
Jan 8 06:46:11.447: INFO: Pod "pod-fd1b0f03-d676-45da-8d9a-f04db4320846": Phase="Pending", Reason="", readiness=false. Elapsed: 176.222458ms
Jan 8 06:46:13.625: INFO: Pod "pod-fd1b0f03-d676-45da-8d9a-f04db4320846": Phase="Pending", Reason="", readiness=false. Elapsed: 2.353932741s
Jan 8 06:46:15.625: INFO: Pod "pod-fd1b0f03-d676-45da-8d9a-f04db4320846": Phase="Pending", Reason="", readiness=false. Elapsed: 4.354154559s
Jan 8 06:46:17.625: INFO: Pod "pod-fd1b0f03-d676-45da-8d9a-f04db4320846": Phase="Pending", Reason="", readiness=false. Elapsed: 6.353913764s
Jan 8 06:46:19.624: INFO: Pod "pod-fd1b0f03-d676-45da-8d9a-f04db4320846": Phase="Pending", Reason="", readiness=false. Elapsed: 8.35308611s
Jan 8 06:46:21.624: INFO: Pod "pod-fd1b0f03-d676-45da-8d9a-f04db4320846": Phase="Running", Reason="", readiness=true. Elapsed: 10.353256919s
Jan 8 06:46:21.624: INFO: Pod "pod-fd1b0f03-d676-45da-8d9a-f04db4320846" satisfied condition "running"
Jan 8 06:46:21.801: INFO: Pod fsgroupchangepolicy-5020/pod-fd1b0f03-d676-45da-8d9a-f04db4320846 started successfully
�[1mSTEP:�[0m Creating a sub-directory and file, and verifying their ownership is 1000 �[38;5;243m01/08/23 06:46:21.801�[0m
Jan 8 06:46:21.802: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/file1] Namespace:fsgroupchangepolicy-5020 PodName:pod-fd1b0f03-d676-45da-8d9a-f04db4320846 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 8 06:46:21.802: INFO: >>> kubeConfig: /root/.kube/config
Jan 8 06:46:21.802: INFO: ExecWithOptions: Clientset creation
Jan 8 06:46:21.802: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-5020/pods/pod-fd1b0f03-d676-45da-8d9a-f04db4320846/exec?command=%2Fbin%2Fsh&command=-c&command=touch+%2Fmnt%2Fvolume1%2Ffile1&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 8 06:46:22.912: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/file1] Namespace:fsgroupchangepolicy-5020 PodName:pod-fd1b0f03-d676-45da-8d9a-f04db4320846 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 8 06:46:22.912: INFO: >>> kubeConfig: /root/.kube/config
Jan 8 06:46:22.913: INFO: ExecWithOptions: Clientset creation
Jan 8 06:46:22.913: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-5020/pods/pod-fd1b0f03-d676-45da-8d9a-f04db4320846/exec?command=%2Fbin%2Fsh&command=-c&command=ls+-l+%2Fmnt%2Fvolume1%2Ffile1&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 8 06:46:24.036: INFO: pod fsgroupchangepolicy-5020/pod-fd1b0f03-d676-45da-8d9a-f04db4320846 exec for cmd ls -l /mnt/volume1/file1, stdout: -rw-r--r-- 1 root 1000 0 Jan 8 06:46 /mnt/volume1/file1, stderr:
Jan 8 06:46:24.036: INFO: stdout split: [-rw-r--r-- 1 root 1000 0 Jan 8 06:46 /mnt/volume1/file1], expected gid: 1000
Jan 8 06:46:24.036: INFO: ExecWithOptions {Command:[/bin/sh -c mkdir /mnt/volume1/subdir] Namespace:fsgroupchangepolicy-5020 PodName:pod-fd1b0f03-d676-45da-8d9a-f04db4320846 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 8 06:46:24.036: INFO: >>> kubeConfig: /root/.kube/config
Jan 8 06:46:24.037: INFO: ExecWithOptions: Clientset creation
Jan 8 06:46:24.037: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-5020/pods/pod-fd1b0f03-d676-45da-8d9a-f04db4320846/exec?command=%2Fbin%2Fsh&command=-c&command=mkdir+%2Fmnt%2Fvolume1%2Fsubdir&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 8 06:46:25.202: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-5020 PodName:pod-fd1b0f03-d676-45da-8d9a-f04db4320846 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 8 06:46:25.202: INFO: >>> kubeConfig: /root/.kube/config
Jan 8 06:46:25.203: INFO: ExecWithOptions: Clientset creation
Jan 8 06:46:25.203: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-5020/pods/pod-fd1b0f03-d676-45da-8d9a-f04db4320846/exec?command=%2Fbin%2Fsh&command=-c&command=touch+%2Fmnt%2Fvolume1%2Fsubdir%2Ffile2&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 8 06:46:26.381: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-5020 PodName:pod-fd1b0f03-d676-45da-8d9a-f04db4320846 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 8 06:46:26.381: INFO: >>> kubeConfig: /root/.kube/config
Jan 8 06:46:26.382: INFO: ExecWithOptions: Clientset creation
Jan 8 06:46:26.382: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-5020/pods/pod-fd1b0f03-d676-45da-8d9a-f04db4320846/exec?command=%2Fbin%2Fsh&command=-c&command=ls+-l+%2Fmnt%2Fvolume1%2Fsubdir%2Ffile2&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 8 06:46:27.541: INFO: pod fsgroupchangepolicy-5020/pod-fd1b0f03-d676-45da-8d9a-f04db4320846 exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r-- 1 root 1000 0 Jan 8 06:46 /mnt/volume1/subdir/file2, stderr:
Jan 8 06:46:27.541: INFO: stdout split: [-rw-r--r-- 1 root 1000 0 Jan 8 06:46 /mnt/volume1/subdir/file2], expected gid: 1000
�[1mSTEP:�[0m Changing the root directory file ownership to 2000 �[38;5;243m01/08/23 06:46:27.541�[0m
Jan 8 06:46:27.541: INFO: ExecWithOptions {Command:[/bin/sh -c chgrp 2000 /mnt/volume1/file1] Namespace:fsgroupchangepolicy-5020 PodName:pod-fd1b0f03-d676-45da-8d9a-f04db4320846 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 8 06:46:27.541: INFO: >>> kubeConfig: /root/.kube/config
Jan 8 06:46:27.542: INFO: ExecWithOptions: Clientset creation
Jan 8 06:46:27.542: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-5020/pods/pod-fd1b0f03-d676-45da-8d9a-f04db4320846/exec?command=%2Fbin%2Fsh&command=-c&command=chgrp+2000+%2Fmnt%2Fvolume1%2Ffile1&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 8 06:46:28.697: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/file1] Namespace:fsgroupchangepolicy-5020 PodName:pod-fd1b0f03-d676-45da-8d9a-f04db4320846 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 8 06:46:28.697: INFO: >>> kubeConfig: /root/.kube/config
Jan 8 06:46:28.698: INFO: ExecWithOptions: Clientset creation
Jan 8 06:46:28.698: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-5020/pods/pod-fd1b0f03-d676-45da-8d9a-f04db4320846/exec?command=%2Fbin%2Fsh&command=-c&command=ls+-l+%2Fmnt%2Fvolume1%2Ffile1&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 8 06:46:29.889: INFO: pod fsgroupchangepolicy-5020/pod-fd1b0f03-d676-45da-8d9a-f04db4320846 exec for cmd ls -l /mnt/volume1/file1, stdout: -rw-r--r-- 1 root 2000 0 Jan 8 06:46 /mnt/volume1/file1, stderr:
Jan 8 06:46:29.889: INFO: stdout split: [-rw-r--r-- 1 root 2000 0 Jan 8 06:46 /mnt/volume1/file1], expected gid: 2000
�[1mSTEP:�[0m Changing the sub-directory file ownership to 3000 �[38;5;243m01/08/23 06:46:29.889�[0m
Jan 8 06:46:29.889: INFO: ExecWithOptions {Command:[/bin/sh -c chgrp 3000 /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-5020 PodName:pod-fd1b0f03-d676-45da-8d9a-f04db4320846 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 8 06:46:29.889: INFO: >>> kubeConfig: /root/.kube/config
Jan 8 06:46:29.891: INFO: ExecWithOptions: Clientset creation
Jan 8 06:46:29.891: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-5020/pods/pod-fd1b0f03-d676-45da-8d9a-f04db4320846/exec?command=%2Fbin%2Fsh&command=-c&command=chgrp+3000+%2Fmnt%2Fvolume1%2Fsubdir%2Ffile2&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 8 06:46:31.014: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-5020 PodName:pod-fd1b0f03-d676-45da-8d9a-f04db4320846 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 8 06:46:31.014: INFO: >>> kubeConfig: /root/.kube/config
Jan 8 06:46:31.015: INFO: ExecWithOptions: Clientset creation
Jan 8 06:46:31.015: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-5020/pods/pod-fd1b0f03-d676-45da-8d9a-f04db4320846/exec?command=%2Fbin%2Fsh&command=-c&command=ls+-l+%2Fmnt%2Fvolume1%2Fsubdir%2Ffile2&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 8 06:46:32.164: INFO: pod fsgroupchangepolicy-5020/pod-fd1b0f03-d676-45da-8d9a-f04db4320846 exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r-- 1 root 3000 0 Jan 8 06:46 /mnt/volume1/subdir/file2, stderr:
Jan 8 06:46:32.164: INFO: stdout split: [-rw-r--r-- 1 root 3000 0 Jan 8 06:46 /mnt/volume1/subdir/file2], expected gid: 3000
�[1mSTEP:�[0m Deleting Pod fsgroupchangepolicy-5020/pod-fd1b0f03-d676-45da-8d9a-f04db4320846 �[38;5;243m01/08/23 06:46:32.165�[0m
Jan 8 06:46:32.165: INFO: Deleting pod "pod-fd1b0f03-d676-45da-8d9a-f04db4320846" in namespace "fsgroupchangepolicy-5020"
Jan 8 06:46:32.343: INFO: Wait up to 5m0s for pod "pod-fd1b0f03-d676-45da-8d9a-f04db4320846" to be fully deleted
�[1mSTEP:�[0m Creating Pod in namespace fsgroupchangepolicy-5020 with fsgroup 4000 �[38;5;243m01/08/23 06:46:34.696�[0m
Jan 8 06:46:34.877: INFO: Waiting up to 5m0s for pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c" in namespace "fsgroupchangepolicy-5020" to be "running"
Jan 8 06:46:35.053: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 176.486499ms
Jan 8 06:46:37.232: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2.355070892s
Jan 8 06:46:39.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4.354346249s
Jan 8 06:46:41.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 6.35430813s
Jan 8 06:46:43.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 8.35307937s
Jan 8 06:46:45.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 10.353410638s
Jan 8 06:46:47.232: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 12.354839698s
Jan 8 06:46:49.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 14.354154934s
Jan 8 06:46:51.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 16.353353989s
Jan 8 06:46:53.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 18.353704704s
Jan 8 06:46:55.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 20.354009038s
Jan 8 06:46:57.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 22.353738468s
Jan 8 06:46:59.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 24.353275229s
Jan 8 06:47:01.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 26.354280419s
Jan 8 06:47:03.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 28.353549921s
Jan 8 06:47:05.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 30.353977567s
Jan 8 06:47:07.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 32.353900583s
Jan 8 06:47:09.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 34.354191796s
Jan 8 06:47:11.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 36.353298586s
Jan 8 06:47:13.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 38.354395834s
Jan 8 06:47:15.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 40.353197434s
Jan 8 06:47:17.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 42.35387325s
Jan 8 06:47:19.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 44.35319297s
Jan 8 06:47:21.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 46.353248099s
Jan 8 06:47:23.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 48.353248757s
Jan 8 06:47:25.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 50.353862689s
Jan 8 06:47:27.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 52.354163257s
Jan 8 06:47:29.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 54.353297661s
Jan 8 06:47:31.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 56.354449828s
Jan 8 06:47:33.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 58.354538286s
Jan 8 06:47:35.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.354389542s
Jan 8 06:47:37.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.354085475s
Jan 8 06:47:39.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.35342069s
Jan 8 06:47:41.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.354221394s
Jan 8 06:47:43.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.353465961s
Jan 8 06:47:45.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.35434005s
Jan 8 06:47:47.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.354032591s
Jan 8 06:47:49.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.354018627s
Jan 8 06:47:51.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.35364432s
Jan 8 06:47:53.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.354146304s
Jan 8 06:47:55.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.353205702s
Jan 8 06:47:57.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.353984969s
Jan 8 06:47:59.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.352980907s
Jan 8 06:48:01.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.354594926s
Jan 8 06:48:03.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.353208927s
Jan 8 06:48:05.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.35437797s
Jan 8 06:48:07.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.353201692s
Jan 8 06:48:09.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.353327403s
Jan 8 06:48:11.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.353533888s
Jan 8 06:48:13.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.353992809s
Jan 8 06:48:15.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.353766484s
Jan 8 06:48:17.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.353977213s
Jan 8 06:48:19.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.353413994s
Jan 8 06:48:21.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.35431886s
Jan 8 06:48:23.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.353423714s
Jan 8 06:48:25.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.354342496s
Jan 8 06:48:27.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.354346873s
Jan 8 06:48:29.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.35417939s
Jan 8 06:48:31.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.353198767s
Jan 8 06:48:33.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.354012135s
Jan 8 06:48:35.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.353276011s
Jan 8 06:48:37.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.353232219s
Jan 8 06:48:39.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.353739337s
Jan 8 06:48:41.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.353434182s
Jan 8 06:48:43.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.353925121s
Jan 8 06:48:45.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.353220454s
Jan 8 06:48:47.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.353258685s
Jan 8 06:48:49.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.354443872s
Jan 8 06:48:51.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.353990065s
Jan 8 06:48:53.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.353979036s
Jan 8 06:48:55.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.353385531s
Jan 8 06:48:57.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.353221072s
Jan 8 06:48:59.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.35325914s
Jan 8 06:49:01.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.353280598s
Jan 8 06:49:03.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.354344716s
Jan 8 06:49:05.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.353983926s
Jan 8 06:49:07.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.35434458s
Jan 8 06:49:09.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.352891187s
Jan 8 06:49:11.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.353188527s
Jan 8 06:49:13.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.354063777s
Jan 8 06:49:15.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.353245354s
Jan 8 06:49:17.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.354204787s
Jan 8 06:49:19.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.353758575s
Jan 8 06:49:21.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.354472927s
Jan 8 06:49:23.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.353330545s
Jan 8 06:49:25.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.354217683s
Jan 8 06:49:27.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.35415815s
Jan 8 06:49:29.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.353248047s
Jan 8 06:49:31.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.353222519s
Jan 8 06:49:33.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.353246527s
Jan 8 06:49:35.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.353435232s
Jan 8 06:49:37.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.353384001s
Jan 8 06:49:39.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.352958509s
Jan 8 06:49:41.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.354312177s
Jan 8 06:49:43.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.352990732s
Jan 8 06:49:45.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.353054656s
Jan 8 06:49:47.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.353844791s
Jan 8 06:49:49.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.353995759s
Jan 8 06:49:51.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.353957299s
Jan 8 06:49:53.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.353260863s
Jan 8 06:49:55.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.353576671s
Jan 8 06:49:57.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.353099516s
Jan 8 06:49:59.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.353261865s
Jan 8 06:50:01.232: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.354798929s
Jan 8 06:50:03.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.353334261s
Jan 8 06:50:05.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.354241475s
Jan 8 06:50:07.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.353638789s
Jan 8 06:50:09.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.354465096s
Jan 8 06:50:11.232: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.354787565s
Jan 8 06:50:13.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.35321475s
Jan 8 06:50:15.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.354155412s
Jan 8 06:50:17.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.353381053s
Jan 8 06:50:19.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.353557575s
Jan 8 06:50:21.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.354461225s
Jan 8 06:50:23.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.353080172s
Jan 8 06:50:25.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.354601035s
Jan 8 06:50:27.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.353799456s
Jan 8 06:50:29.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.353911305s
Jan 8 06:50:31.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.354631948s
Jan 8 06:50:33.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.354486027s
Jan 8 06:50:35.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.354157607s
Jan 8 06:50:37.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.354270646s
Jan 8 06:50:39.233: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.356223103s
Jan 8 06:50:41.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.353982046s
Jan 8 06:50:43.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.353099473s
Jan 8 06:50:45.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.353414672s
Jan 8 06:50:47.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.353366347s
Jan 8 06:50:49.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.354236041s
Jan 8 06:50:51.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.353538559s
Jan 8 06:50:53.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.353848138s
Jan 8 06:50:55.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.353869524s
Jan 8 06:50:57.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.353884185s
Jan 8 06:50:59.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.352951896s
Jan 8 06:51:01.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.353433177s
Jan 8 06:51:03.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.354352583s
Jan 8 06:51:05.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.354109341s
Jan 8 06:51:07.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.35316401s
Jan 8 06:51:09.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.354373916s
Jan 8 06:51:11.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.353265673s
Jan 8 06:51:13.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.353544068s
Jan 8 06:51:15.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.354576387s
Jan 8 06:51:17.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.353341333s
Jan 8 06:51:19.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.35414868s
Jan 8 06:51:21.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.354428789s
Jan 8 06:51:23.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.353859632s
Jan 8 06:51:25.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.353623983s
Jan 8 06:51:27.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.354493034s
Jan 8 06:51:29.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.354276943s
Jan 8 06:51:31.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.35437361s
Jan 8 06:51:33.231: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.354139212s
Jan 8 06:51:35.230: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.353538609s
Jan 8 06:51:35.407: INFO: Pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.530203916s
Jan 8 06:51:35.407: INFO: Unexpected error:
<*errors.errorString | 0xc00107fe80>: {
s: "pod \"pod-a15c4150-c88e-49b1-a111-7f2271cf819c\" is not Running: timed out while waiting for pod fsgroupchangepolicy-5020/pod-a15c4150-c88e-49b1-a111-7f2271cf819c to be running",
}
Jan 8 06:51:35.407: FAIL: pod "pod-a15c4150-c88e-49b1-a111-7f2271cf819c" is not Running: timed out while waiting for pod fsgroupchangepolicy-5020/pod-a15c4150-c88e-49b1-a111-7f2271cf819c to be running
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc001420f00, 0xc000f43e10, 0x0, {0xc004b5be10, 0x4}, {0xc004b5be14, 0x4})
test/e2e/storage/testsuites/fsgroupchangepolicy.go:258 +0x151
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3()
test/e2e/storage/testsuites/fsgroupchangepolicy.go:247 +0x556
�[1mSTEP:�[0m Deleting pvc �[38;5;243m01/08/23 06:51:35.407�[0m
Jan 8 06:51:35.760: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comzdq74"
Jan 8 06:51:35.940: INFO: Waiting up to 5m0s for PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 to get deleted
Jan 8 06:51:36.116: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (176.312753ms)
Jan 8 06:51:41.293: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (5.353157399s)
Jan 8 06:51:46.470: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (10.530678605s)
Jan 8 06:51:51.647: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (15.707234757s)
Jan 8 06:51:56.824: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (20.88454527s)
Jan 8 06:52:02.002: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (26.062106897s)
Jan 8 06:52:07.180: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (31.240000494s)
Jan 8 06:52:12.357: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (36.416919975s)
Jan 8 06:52:17.533: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (41.593716953s)
Jan 8 06:52:22.711: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (46.771272767s)
Jan 8 06:52:27.887: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (51.947848319s)
Jan 8 06:52:33.067: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (57.127436435s)
Jan 8 06:52:38.244: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (1m2.304128423s)
Jan 8 06:52:43.420: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (1m7.480807965s)
Jan 8 06:52:48.597: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (1m12.657572835s)
Jan 8 06:52:53.774: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (1m17.834185048s)
Jan 8 06:52:58.950: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (1m23.010734141s)
Jan 8 06:53:04.128: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (1m28.18846685s)
Jan 8 06:53:09.305: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (1m33.365535881s)
Jan 8 06:53:14.482: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (1m38.542407567s)
Jan 8 06:53:19.660: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (1m43.720561203s)
Jan 8 06:53:24.837: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (1m48.897882393s)
Jan 8 06:53:30.015: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (1m54.075527111s)
Jan 8 06:53:35.192: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (1m59.25235692s)
Jan 8 06:53:40.368: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (2m4.428880246s)
Jan 8 06:53:45.546: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (2m9.605908274s)
Jan 8 06:53:50.724: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (2m14.784390156s)
Jan 8 06:53:55.902: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (2m19.96196765s)
Jan 8 06:54:01.079: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (2m25.139136541s)
Jan 8 06:54:06.256: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (2m30.316423026s)
Jan 8 06:54:11.433: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (2m35.493374889s)
Jan 8 06:54:16.611: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (2m40.671109061s)
Jan 8 06:54:21.787: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (2m45.847845042s)
Jan 8 06:54:26.965: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (2m51.024931474s)
Jan 8 06:54:32.141: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (2m56.201759293s)
Jan 8 06:54:37.319: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (3m1.379372393s)
Jan 8 06:54:42.495: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (3m6.555678969s)
Jan 8 06:54:47.672: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (3m11.732725027s)
Jan 8 06:54:52.852: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (3m16.912185797s)
Jan 8 06:54:58.028: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (3m22.088748568s)
Jan 8 06:55:03.206: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (3m27.26638982s)
Jan 8 06:55:08.384: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (3m32.443938513s)
Jan 8 06:55:13.561: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (3m37.621310948s)
Jan 8 06:55:18.737: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (3m42.797789776s)
Jan 8 06:55:23.915: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (3m47.975120416s)
Jan 8 06:55:29.091: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (3m53.151399275s)
Jan 8 06:55:34.268: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (3m58.328361127s)
Jan 8 06:55:39.452: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (4m3.512375627s)
Jan 8 06:55:44.629: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (4m8.688930073s)
Jan 8 06:55:49.806: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (4m13.866369128s)
Jan 8 06:55:54.983: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (4m19.043711209s)
Jan 8 06:56:00.161: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (4m24.221109225s)
Jan 8 06:56:05.338: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (4m29.397996647s)
Jan 8 06:56:10.515: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (4m34.575163049s)
Jan 8 06:56:15.693: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (4m39.75296937s)
Jan 8 06:56:20.869: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (4m44.929666245s)
Jan 8 06:56:26.046: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (4m50.106375822s)
Jan 8 06:56:31.222: INFO: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 found and phase=Bound (4m55.282546155s)
�[1mSTEP:�[0m Deleting sc �[38;5;243m01/08/23 06:56:36.223�[0m
Jan 8 06:56:36.401: INFO: Unexpected error: while cleanup resource:
<errors.aggregate | len:1, cap:1>: [
<errors.aggregate | len:1, cap:1>[
<*fmt.wrapError | 0xc00390a060>{
msg: "persistent Volume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 not deleted by dynamic provisioner: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 still exists within 5m0s",
err: <*errors.errorString | 0xc000e8af40>{
s: "PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 still exists within 5m0s",
},
},
],
]
Jan 8 06:56:36.401: FAIL: while cleanup resource: persistent Volume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 not deleted by dynamic provisioner: PersistentVolume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07 still exists within 5m0s
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2()
test/e2e/storage/testsuites/fsgroupchangepolicy.go:128 +0x14e
panic({0x70eb680, 0xc00072c620})
/usr/local/go/src/runtime/panic.go:884 +0x212
k8s.io/kubernetes/test/e2e/framework.Fail({0xc00274a000, 0xae}, {0xc00153fc30?, 0xc00274a000?, 0xc00153fc58?})
test/e2e/framework/log.go:61 +0x145
k8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0x1, {0x7fa4200, 0xc00107fe80}, {0x0?, 0x2?, 0x0?})
test/e2e/framework/expect.go:76 +0x267
k8s.io/kubernetes/test/e2e/framework.ExpectNoError(...)
test/e2e/framework/expect.go:43
k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc001420f00, 0xc000f43e10, 0x0, {0xc004b5be10, 0x4}, {0xc004b5be14, 0x4})
test/e2e/storage/testsuites/fsgroupchangepolicy.go:258 +0x151
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3()
test/e2e/storage/testsuites/fsgroupchangepolicy.go:247 +0x556
[AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
test/e2e/framework/node/init/init.go:32
Jan 8 06:56:36.402: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/08/23 06:56:36.58�[0m
�[1mSTEP:�[0m Collecting events from namespace "fsgroupchangepolicy-5020". �[38;5;243m01/08/23 06:56:36.58�[0m
�[1mSTEP:�[0m Found 21 events. �[38;5;243m01/08/23 06:56:36.757�[0m
Jan 8 06:56:36.757: INFO: At 2023-01-08 06:46:10 +0000 UTC - event for ebs.csi.aws.comzdq74: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Jan 8 06:56:36.757: INFO: At 2023-01-08 06:46:11 +0000 UTC - event for ebs.csi.aws.comzdq74: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator
Jan 8 06:56:36.757: INFO: At 2023-01-08 06:46:11 +0000 UTC - event for ebs.csi.aws.comzdq74: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-vzzj9_eb492b31-b783-48e6-ab2b-bc22df496b4f } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-5020/ebs.csi.aws.comzdq74"
Jan 8 06:56:36.757: INFO: At 2023-01-08 06:46:14 +0000 UTC - event for ebs.csi.aws.comzdq74: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-vzzj9_eb492b31-b783-48e6-ab2b-bc22df496b4f } ProvisioningSucceeded: Successfully provisioned volume pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07
Jan 8 06:56:36.757: INFO: At 2023-01-08 06:46:15 +0000 UTC - event for pod-fd1b0f03-d676-45da-8d9a-f04db4320846: {default-scheduler } Scheduled: Successfully assigned fsgroupchangepolicy-5020/pod-fd1b0f03-d676-45da-8d9a-f04db4320846 to i-0bd196cab651da520
Jan 8 06:56:36.757: INFO: At 2023-01-08 06:46:17 +0000 UTC - event for pod-fd1b0f03-d676-45da-8d9a-f04db4320846: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-c52cd1ca-f1c7-4094-9c97-7b94d0962b07"
Jan 8 06:56:36.757: INFO: At 2023-01-08 06:46:19 +0000 UTC - event for pod-fd1b0f03-d676-45da-8d9a-f04db4320846: {kubelet i-0bd196cab651da520} Started: Started container write-pod
Jan 8 06:56:36.757: INFO: At 2023-01-08 06:46:19 +0000 UTC - event for pod-fd1b0f03-d676-45da-8d9a-f04db4320846: {kubelet i-0bd196cab651da520} Created: Created container write-pod
Jan 8 06:56:36.757: INFO: At 2023-01-08 06:46:19 +0000 UTC - event for pod-fd1b0f03-d676-45da-8d9a-f04db4320846: {kubelet i-0bd196cab651da520} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine
Jan 8 06:56:36.757: INFO: At 2023-01-08 06:46:32 +0000 UTC - event for pod-fd1b0f03-d676-45da-8d9a-f04db4320846: {kubelet i-0bd196cab651da520} Killing: Stopping container write-pod
Jan 8 06:56:36.757: INFO: At 2023-01-08 06:46:34 +0000 UTC - event for pod-a15c4150-c88e-49b1-a111-7f2271cf819c: {default-scheduler } Scheduled: Successfully assigned fsgroupchangepolicy-5020/pod-a15c4150-c88e-49b1-a111-7f2271cf819c to i-0bd196cab651da520
Jan 8 06:56:36.757: INFO: At 2023-01-08 06:46:39 +0000 UTC - event for pod-a15c4150-c88e-49b1-a111-7f2271cf819c: {kubelet i-0bd196cab651da520} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d915a485992024f7fb5970edfe7b9264ba0073097a6babe66f585098544ef328": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:56:36.758: INFO: At 2023-01-08 06:46:50 +0000 UTC - event for pod-a15c4150-c88e-49b1-a111-7f2271cf819c: {kubelet i-0bd196cab651da520} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d0aee5a9cb53a4ce0898c80a901ef3384a576cc7a6cd2904ba23f10ba59b9933": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:56:36.758: INFO: At 2023-01-08 06:47:02 +0000 UTC - event for pod-a15c4150-c88e-49b1-a111-7f2271cf819c: {kubelet i-0bd196cab651da520} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d6e07b1d3adc171a8666b5e3bd9c6022b342caa551c81cdacfaa6b54764445b0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:56:36.758: INFO: At 2023-01-08 06:47:16 +0000 UTC - event for pod-a15c4150-c88e-49b1-a111-7f2271cf819c: {kubelet i-0bd196cab651da520} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "449019cd85c2f67fad52cc66530f50da97159806ecd96e40840d19e2193e5b98": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:56:36.758: INFO: At 2023-01-08 06:47:27 +0000 UTC - event for pod-a15c4150-c88e-49b1-a111-7f2271cf819c: {kubelet i-0bd196cab651da520} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "657f271085fd063a64acc4c23e047dfd3c51c6f262c6d390ecd25913e857d078": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:56:36.758: INFO: At 2023-01-08 06:47:40 +0000 UTC - event for pod-a15c4150-c88e-49b1-a111-7f2271cf819c: {kubelet i-0bd196cab651da520} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7e47a12bd5163bcd35aa6b3290678a8fbecfa2e7bc31d2862dc1abe2f005c04b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:56:36.758: INFO: At 2023-01-08 06:47:51 +0000 UTC - event for pod-a15c4150-c88e-49b1-a111-7f2271cf819c: {kubelet i-0bd196cab651da520} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "956ff9cfeff99337cd2e8c403af439507cdd2615d4a8e2b6476d879f08d30290": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:56:36.758: INFO: At 2023-01-08 06:48:03 +0000 UTC - event for pod-a15c4150-c88e-49b1-a111-7f2271cf819c: {kubelet i-0bd196cab651da520} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "be154e9d2075833448a15061575f4132856ada055ad9822142e0480c9ae06432": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:56:36.758: INFO: At 2023-01-08 06:48:15 +0000 UTC - event for pod-a15c4150-c88e-49b1-a111-7f2271cf819c: {kubelet i-0bd196cab651da520} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e4b6b26f365e972580fcbcb73dcf8c1d97c69c97dce7c0f1a733b106902d2f9b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:56:36.758: INFO: At 2023-01-08 06:48:30 +0000 UTC - event for pod-a15c4150-c88e-49b1-a111-7f2271cf819c: {kubelet i-0bd196cab651da520} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "16cd17e69220aee360bec937fb02ceb8e70cd547b479bcc744475d07bee2549e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:56:36.934: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 8 06:56:36.934: INFO: pod-a15c4150-c88e-49b1-a111-7f2271cf819c i-0bd196cab651da520 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:34 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:52:41 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:52:41 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:34 +0000 UTC }]
Jan 8 06:56:36.934: INFO:
Jan 8 06:56:37.290: INFO:
Logging node info for node i-0685adc783a822be6
Jan 8 06:56:37.466: INFO: Node Info: &Node{ObjectMeta:{i-0685adc783a822be6 3eb588c7-cc76-44b4-b28a-492b9989254b 43426 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0685adc783a822be6 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0685adc783a822be6 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.112 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-8725":"i-0685adc783a822be6","ebs.csi.aws.com":"i-0685adc783a822be6"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:55:45 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-08 06:55:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0685adc783a822be6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:45 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:45 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:45 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:55:45 +0000 UTC,LastTransitionTime:2023-01-08 06:25:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.112,},NodeAddress{Type:ExternalIP,Address:43.201.46.149,},NodeAddress{Type:InternalDNS,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-201-46-149.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dccafbf198926b990228825ae0608,SystemUUID:ec2dccaf-bf19-8926-b990-228825ae0608,BootID:b8ff627c-af94-4730-be1d-6c7bfbfc818f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0488b0c43122e11c1 kubernetes.io/csi/ebs.csi.aws.com^vol-0b8657a4eb6501329],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0b8657a4eb6501329,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0488b0c43122e11c1,DevicePath:,},},Config:nil,},}
Jan 8 06:56:37.467: INFO:
Logging kubelet events for node i-0685adc783a822be6
Jan 8 06:56:37.645: INFO:
Logging pods the kubelet thinks is on node i-0685adc783a822be6
Jan 8 06:56:37.833: INFO: agnhost-primary-9vrbd started at 2023-01-08 06:54:54 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container agnhost-primary ready: false, restart count 0
Jan 8 06:56:37.833: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-97qp6 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:37.833: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-4d46c started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:37.833: INFO: csi-mockplugin-0 started at 2023-01-08 06:51:13 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container csi-provisioner ready: false, restart count 0
Jan 8 06:56:37.833: INFO: Container driver-registrar ready: false, restart count 0
Jan 8 06:56:37.833: INFO: Container mock ready: false, restart count 0
Jan 8 06:56:37.833: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-zgmck started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:37.833: INFO: pod-disruption-failure-ignore-1-kfqqr started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container c ready: false, restart count 0
Jan 8 06:56:37.833: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-gq5pr started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:37.833: INFO: inline-volume-tester-9qdkl started at 2023-01-08 06:52:11 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 8 06:56:37.833: INFO: csi-mockplugin-0 started at 2023-01-08 06:50:39 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container csi-provisioner ready: true, restart count 0
Jan 8 06:56:37.833: INFO: Container driver-registrar ready: true, restart count 0
Jan 8 06:56:37.833: INFO: Container mock ready: true, restart count 0
Jan 8 06:56:37.833: INFO: cilium-9vhbx started at 2023-01-08 06:25:38 +0000 UTC (1+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 8 06:56:37.833: INFO: Container cilium-agent ready: true, restart count 0
Jan 8 06:56:37.833: INFO: pod-937b6812-ddb0-47fe-af8b-07288c742e73 started at 2023-01-08 06:52:10 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container write-pod ready: false, restart count 0
Jan 8 06:56:37.833: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-svbp2 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:37.833: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mpw58 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:37.833: INFO: service-proxy-toggled-dh67n started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container service-proxy-toggled ready: false, restart count 0
Jan 8 06:56:37.833: INFO: liveness-51325787-aaa2-4ac1-a3f9-8ff47e3fa793 started at 2023-01-08 06:54:56 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container agnhost-container ready: false, restart count 0
Jan 8 06:56:37.833: INFO: pod-subpath-test-inlinevolume-plpm started at 2023-01-08 06:52:37 +0000 UTC (2+2 container statuses recorded)
Jan 8 06:56:37.833: INFO: Init container init-volume-inlinevolume-plpm ready: false, restart count 0
Jan 8 06:56:37.833: INFO: Init container test-init-subpath-inlinevolume-plpm ready: false, restart count 0
Jan 8 06:56:37.833: INFO: Container test-container-subpath-inlinevolume-plpm ready: false, restart count 0
Jan 8 06:56:37.833: INFO: Container test-container-volume-inlinevolume-plpm ready: false, restart count 0
Jan 8 06:56:37.833: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:51:13 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container csi-attacher ready: false, restart count 0
Jan 8 06:56:37.833: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rrmzm started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:37.833: INFO: pod-disruption-failure-ignore-0-w4rvn started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container c ready: false, restart count 0
Jan 8 06:56:37.833: INFO: pod-projected-secrets-907df958-d79f-42f9-87ee-41301a6f40be started at 2023-01-08 06:55:13 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container secret-volume-test ready: false, restart count 0
Jan 8 06:56:37.833: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:50:39 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container csi-attacher ready: false, restart count 0
Jan 8 06:56:37.833: INFO: suspend-false-to-true-97wd6 started at 2023-01-08 06:53:18 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container c ready: false, restart count 0
Jan 8 06:56:37.833: INFO: csi-mockplugin-0 started at 2023-01-08 06:56:04 +0000 UTC (0+4 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container busybox ready: false, restart count 0
Jan 8 06:56:37.833: INFO: Container csi-provisioner ready: false, restart count 0
Jan 8 06:56:37.833: INFO: Container driver-registrar ready: false, restart count 0
Jan 8 06:56:37.833: INFO: Container mock ready: false, restart count 0
Jan 8 06:56:37.833: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-q97zd started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:37.833: INFO: ebs-csi-node-ft7bw started at 2023-01-08 06:25:38 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:56:37.833: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:56:37.833: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 8 06:56:37.833: INFO: service-proxy-disabled-6pp8x started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container service-proxy-disabled ready: true, restart count 0
Jan 8 06:56:37.833: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2bn4h started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:37.833: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v9tjx started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:37.833: INFO: affinity-clusterip-zxklh started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:37.833: INFO: Container affinity-clusterip ready: false, restart count 0
Jan 8 06:56:38.466: INFO:
Latency metrics for node i-0685adc783a822be6
Jan 8 06:56:38.466: INFO:
Logging node info for node i-0bd196cab651da520
Jan 8 06:56:38.643: INFO: Node Info: &Node{ObjectMeta:{i-0bd196cab651da520 f7f00780-d24e-4c8e-b54d-373598e1d1d2 42453 0 2023-01-08 06:26:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0bd196cab651da520 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0bd196cab651da520 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.104 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bd196cab651da520"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:26:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:52:47 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-08 06:52:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0bd196cab651da520,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:52:46 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:52:46 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:52:46 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:52:46 +0000 UTC,LastTransitionTime:2023-01-08 06:26:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.104,},NodeAddress{Type:ExternalIP,Address:54.180.1.55,},NodeAddress{Type:InternalDNS,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-180-1-55.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec218f9d32a84b2feeb6301255fd576c,SystemUUID:ec218f9d-32a8-4b2f-eeb6-301255fd576c,BootID:accdf7fd-1195-4765-b28e-29d44bf47bac,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-06c6e766891904b0f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-06c6e766891904b0f,DevicePath:,},},Config:nil,},}
Jan 8 06:56:38.643: INFO:
Logging kubelet events for node i-0bd196cab651da520
Jan 8 06:56:38.821: INFO:
Logging pods the kubelet thinks is on node i-0bd196cab651da520
Jan 8 06:56:39.005: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-75r8w started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.005: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:39.005: INFO: pod-a15c4150-c88e-49b1-a111-7f2271cf819c started at 2023-01-08 06:46:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.005: INFO: Container write-pod ready: true, restart count 0
Jan 8 06:56:39.005: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v44bk started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.005: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:39.005: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9mp69 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.005: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:39.005: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-cddx5 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.005: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:39.005: INFO: suspend-false-to-true-xwpqd started at 2023-01-08 06:53:18 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.005: INFO: Container c ready: false, restart count 0
Jan 8 06:56:39.005: INFO: pod6b22b1b4-d027-4092-9a22-c1f06499be00 started at 2023-01-08 06:56:02 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.005: INFO: Container nginx ready: false, restart count 0
Jan 8 06:56:39.005: INFO: pod-projected-configmaps-143d6d2e-7b9b-47e8-97c7-c2b8adc28191 started at 2023-01-08 06:52:29 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.005: INFO: Container agnhost-container ready: false, restart count 0
Jan 8 06:56:39.005: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-6df8l started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.005: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:39.005: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-vrcjq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.005: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:39.005: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9776c started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.005: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:39.005: INFO: busybox-scheduling-0abca61c-e823-4136-8618-ce7deb358c0b started at 2023-01-08 06:55:28 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.005: INFO: Container busybox-scheduling-0abca61c-e823-4136-8618-ce7deb358c0b ready: false, restart count 0
Jan 8 06:56:39.005: INFO: ebs-csi-node-6grlm started at 2023-01-08 06:26:02 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:56:39.005: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:56:39.005: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:56:39.005: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 8 06:56:39.006: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9wskq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.006: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:39.006: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mphgx started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.006: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:39.006: INFO: pod1 started at 2023-01-08 06:51:10 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.006: INFO: Container agnhost-container ready: true, restart count 0
Jan 8 06:56:39.006: INFO: affinity-clusterip-mrptn started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.006: INFO: Container affinity-clusterip ready: false, restart count 0
Jan 8 06:56:39.006: INFO: cilium-s6shx started at 2023-01-08 06:26:02 +0000 UTC (1+1 container statuses recorded)
Jan 8 06:56:39.006: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 8 06:56:39.006: INFO: Container cilium-agent ready: true, restart count 0
Jan 8 06:56:39.006: INFO: service-proxy-toggled-8nfkz started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.006: INFO: Container service-proxy-toggled ready: false, restart count 0
Jan 8 06:56:39.006: INFO: execpodgmcm4 started at 2023-01-08 06:55:35 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.006: INFO: Container agnhost-container ready: false, restart count 0
Jan 8 06:56:39.006: INFO: sample-webhook-deployment-865554f4d9-4kppl started at 2023-01-08 06:55:41 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.006: INFO: Container sample-webhook ready: false, restart count 0
Jan 8 06:56:39.006: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-76nxw started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:39.006: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:39.637: INFO:
Latency metrics for node i-0bd196cab651da520
Jan 8 06:56:39.637: INFO:
Logging node info for node i-0d149644e2edcf9cf
Jan 8 06:56:39.813: INFO: Node Info: &Node{ObjectMeta:{i-0d149644e2edcf9cf b448ef72-6e61-4ad5-aa29-d63cc0f94945 43645 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0d149644e2edcf9cf kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0d149644e2edcf9cf topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.115 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0d149644e2edcf9cf"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:56:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0d149644e2edcf9cf,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.115,},NodeAddress{Type:ExternalIP,Address:3.35.219.218,},NodeAddress{Type:InternalDNS,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-35-219-218.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec26c6a9af66729e278545162595eb2d,SystemUUID:ec26c6a9-af66-729e-2785-45162595eb2d,BootID:959da9e3-fb86-442a-958f-2a49c509cdf4,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 8 06:56:39.814: INFO:
Logging kubelet events for node i-0d149644e2edcf9cf
Jan 8 06:56:39.992: INFO:
Logging pods the kubelet thinks is on node i-0d149644e2edcf9cf
Jan 8 06:56:40.192: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:52:13 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container csi-attacher ready: false, restart count 0
Jan 8 06:56:40.192: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9b2xq started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:40.192: INFO: probe-test-8a476652-ee98-415a-b6f6-009d080c5f05 started at 2023-01-08 06:51:44 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container probe-test-8a476652-ee98-415a-b6f6-009d080c5f05 ready: false, restart count 0
Jan 8 06:56:40.192: INFO: csi-mockplugin-0 started at 2023-01-08 06:52:13 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container csi-provisioner ready: false, restart count 0
Jan 8 06:56:40.192: INFO: Container driver-registrar ready: false, restart count 0
Jan 8 06:56:40.192: INFO: Container mock ready: false, restart count 0
Jan 8 06:56:40.192: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-njh2f started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:40.192: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-flhps started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:40.192: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2vh6v started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:40.192: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nt9jr started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0
Jan 8 06:56:40.192: INFO: cilium-58twl started at 2023-01-08 06:25:37 +0000 UTC (1+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 8 06:56:40.192: INFO: Container cilium-agent ready: true, restart count 0
Jan 8 06:56:40.192: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rv5qq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:40.192: INFO: csi-hostpathplugin-0 started at 2023-01-08 06:52:05 +0000 UTC (0+7 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container csi-attacher ready: false, restart count 0
Jan 8 06:56:40.192: INFO: Container csi-provisioner ready: false, restart count 0
Jan 8 06:56:40.192: INFO: Container csi-resizer ready: false, restart count 0
Jan 8 06:56:40.192: INFO: Container csi-snapshotter ready: false, restart count 0
Jan 8 06:56:40.192: INFO: Container hostpath ready: false, restart count 0
Jan 8 06:56:40.192: INFO: Container liveness-probe ready: false, restart count 0
Jan 8 06:56:40.192: INFO: Container node-driver-registrar ready: false, restart count 0
Jan 8 06:56:40.192: INFO: ebs-csi-node-lfcwb started at 2023-01-08 06:25:37 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:56:40.192: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:56:40.192: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 8 06:56:40.192: INFO: coredns-559769c974-9q4qg started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container coredns ready: true, restart count 0
Jan 8 06:56:40.192: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nsfnm started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:40.192: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-xrmq4 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:40.192: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-7gwsq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:40.192: INFO: service-proxy-disabled-v5f82 started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container service-proxy-disabled ready: true, restart count 0
Jan 8 06:56:40.192: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-tgn6r started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:40.192: INFO: coredns-autoscaler-7cb5c5b969-xqnk5 started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:40.192: INFO: Container autoscaler ready: true, restart count 0
Jan 8 06:56:40.825: INFO:
Latency metrics for node i-0d149644e2edcf9cf
Jan 8 06:56:40.825: INFO:
Logging node info for node i-0eee6113fd9f81d94
Jan 8 06:56:41.002: INFO: Node Info: &Node{ObjectMeta:{i-0eee6113fd9f81d94 6de592fc-f11c-4ba3-afa0-707bb4ca29b2 41943 0 2023-01-08 06:23:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0eee6113fd9f81d94 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.48.64 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eee6113fd9f81d94"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-08 06:23:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-08 06:24:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-08 06:24:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-08 06:51:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0eee6113fd9f81d94,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:51:55 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:51:55 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:51:55 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:51:55 +0000 UTC,LastTransitionTime:2023-01-08 06:24:35 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.48.64,},NodeAddress{Type:ExternalIP,Address:3.35.206.95,},NodeAddress{Type:InternalDNS,Address:i-0eee6113fd9f81d94.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0eee6113fd9f81d94.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-35-206-95.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d8cf21861b15bb593210ff59f6044,SystemUUID:ec2d8cf2-1861-b15b-b593-210ff59f6044,BootID:fcad947c-a63f-4722-9a63-22eac74bea9f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43453935,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42803460,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967354,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 8 06:56:41.002: INFO:
Logging kubelet events for node i-0eee6113fd9f81d94
Jan 8 06:56:41.180: INFO:
Logging pods the kubelet thinks is on node i-0eee6113fd9f81d94
Jan 8 06:56:41.362: INFO: kube-controller-manager-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:41.363: INFO: Container kube-controller-manager ready: true, restart count 2
Jan 8 06:56:41.363: INFO: ebs-csi-node-nspqr started at 2023-01-08 06:24:21 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:56:41.363: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:56:41.363: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:56:41.363: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 8 06:56:41.363: INFO: aws-cloud-controller-manager-d8d56 started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:41.363: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 8 06:56:41.363: INFO: cilium-operator-5dd44dc49f-fd8vl started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:41.363: INFO: Container cilium-operator ready: true, restart count 0
Jan 8 06:56:41.363: INFO: ebs-csi-controller-89b649c94-vzzj9 started at 2023-01-08 06:24:21 +0000 UTC (0+5 container statuses recorded)
Jan 8 06:56:41.363: INFO: Container csi-attacher ready: true, restart count 0
Jan 8 06:56:41.363: INFO: Container csi-provisioner ready: true, restart count 0
Jan 8 06:56:41.363: INFO: Container csi-resizer ready: true, restart count 0
Jan 8 06:56:41.363: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:56:41.363: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:56:41.363: INFO: dns-controller-69987775c6-v4cmg started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:41.363: INFO: Container dns-controller ready: true, restart count 0
Jan 8 06:56:41.363: INFO: kube-scheduler-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:41.363: INFO: Container kube-scheduler ready: true, restart count 0
Jan 8 06:56:41.363: INFO: kube-apiserver-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+2 container statuses recorded)
Jan 8 06:56:41.363: INFO: Container healthcheck ready: true, restart count 0
Jan 8 06:56:41.363: INFO: Container kube-apiserver ready: true, restart count 1
Jan 8 06:56:41.363: INFO: kops-controller-242qq started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:41.363: INFO: Container kops-controller ready: true, restart count 0
Jan 8 06:56:41.363: INFO: cilium-f22px started at 2023-01-08 06:24:21 +0000 UTC (1+1 container statuses recorded)
Jan 8 06:56:41.363: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 8 06:56:41.363: INFO: Container cilium-agent ready: true, restart count 0
Jan 8 06:56:41.363: INFO: etcd-manager-events-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:41.363: INFO: Container etcd-manager ready: true, restart count 0
Jan 8 06:56:41.363: INFO: etcd-manager-main-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:41.363: INFO: Container etcd-manager ready: true, restart count 0
Jan 8 06:56:41.967: INFO:
Latency metrics for node i-0eee6113fd9f81d94
Jan 8 06:56:41.967: INFO:
Logging node info for node i-0f469f08be563dad4
Jan 8 06:56:42.144: INFO: Node Info: &Node{ObjectMeta:{i-0f469f08be563dad4 fddb222c-a6ac-415f-83dc-4b71acb0e5ab 43138 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0f469f08be563dad4 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0f469f08be563dad4 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.57.173 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0f469f08be563dad4"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:55:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0f469f08be563dad4,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.57.173,},NodeAddress{Type:ExternalIP,Address:3.38.152.241,},NodeAddress{Type:InternalDNS,Address:i-0f469f08be563dad4.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f469f08be563dad4.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-38-152-241.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2abc7b1d816532cfb4787d85cb98ea,SystemUUID:ec2abc7b-1d81-6532-cfb4-787d85cb98ea,BootID:442daa6f-46d1-4c32-9182-8b6204c22aba,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 8 06:56:42.144: INFO:
Logging kubelet events for node i-0f469f08be563dad4
Jan 8 06:56:42.323: INFO:
Logging pods the kubelet thinks is on node i-0f469f08be563dad4
Jan 8 06:56:42.504: INFO: cilium-24rbm started at 2023-01-08 06:25:37 +0000 UTC (1+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 8 06:56:42.504: INFO: Container cilium-agent ready: true, restart count 0
Jan 8 06:56:42.504: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-ktkbb started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:42.504: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-dvpgx started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:42.504: INFO: ebs-csi-node-9s8qc started at 2023-01-08 06:25:37 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:56:42.504: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:56:42.504: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 8 06:56:42.504: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-x88cb started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:42.504: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-h5c5c started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:42.504: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-jc58p started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:42.504: INFO: service-proxy-disabled-bqbxv started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container service-proxy-disabled ready: true, restart count 0
Jan 8 06:56:42.504: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-gdlbp started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:42.504: INFO: service-proxy-toggled-7qcjj started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container service-proxy-toggled ready: true, restart count 0
Jan 8 06:56:42.504: INFO: coredns-559769c974-f9s4x started at 2023-01-08 06:26:19 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container coredns ready: true, restart count 0
Jan 8 06:56:42.504: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-glvzd started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:42.504: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-cwg4t started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:42.504: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-57rwf started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0
Jan 8 06:56:42.504: INFO: pod-configmaps-393a999b-3a18-4df6-95b4-26106a4b0e61 started at 2023-01-08 06:51:56 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container agnhost-container ready: false, restart count 0
Jan 8 06:56:42.504: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-b6m8x started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:56:42.504: INFO: affinity-clusterip-8kclb started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:56:42.504: INFO: Container affinity-clusterip ready: false, restart count 0
Jan 8 06:56:43.169: INFO:
Latency metrics for node i-0f469f08be563dad4
[DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "fsgroupchangepolicy-5020" for this suite. �[38;5;243m01/08/23 06:56:43.169�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(delayed\sbinding\)\]\stopology\sshould\sprovision\sa\svolume\sand\sschedule\sa\spod\swith\sAllowedTopologies$'
test/e2e/storage/testsuites/topology.go:179 k8s.io/kubernetes/test/e2e/storage/testsuites.(*topologyTestSuite).DefineTests.func3() test/e2e/storage/testsuites/topology.go:179 +0x4b5from junit_01.xml
[BeforeEach] [Testpattern: Dynamic PV (delayed binding)] topology test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (delayed binding)] topology set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/08/23 06:52:04.889�[0m Jan 8 06:52:04.889: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename topology �[38;5;243m01/08/23 06:52:04.89�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/08/23 06:52:05.427�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/08/23 06:52:05.782�[0m [BeforeEach] [Testpattern: Dynamic PV (delayed binding)] topology test/e2e/framework/metrics/init/init.go:31 [It] should provision a volume and schedule a pod with AllowedTopologies test/e2e/storage/testsuites/topology.go:163 Jan 8 06:52:06.317: INFO: found topology map[topology.ebs.csi.aws.com/zone:ap-northeast-2a] Jan 8 06:52:06.317: INFO: Creating storage class object and pvc object for driver - sc: &StorageClass{ObjectMeta:{topology-8910-e2e-sckqrdp 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Provisioner:ebs.csi.aws.com,Parameters:map[string]string{encrypted: true,type: gp3,},ReclaimPolicy:nil,MountOptions:[],AllowVolumeExpansion:*true,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{{[{topology.ebs.csi.aws.com/zone [ap-northeast-2a]}]},},}, pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- topology-8910 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},Claims:[]ResourceClaim{},},VolumeName:,Selector:nil,StorageClassName:*topology-8910-e2e-sckqrdp,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},} �[1mSTEP:�[0m Creating sc �[38;5;243m01/08/23 06:52:06.317�[0m �[1mSTEP:�[0m Creating pvc �[38;5;243m01/08/23 06:52:06.497�[0m �[1mSTEP:�[0m Creating pod �[38;5;243m01/08/23 06:52:06.677�[0m Jan 8 06:52:06.858: INFO: Waiting up to 5m0s for pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73" in namespace "topology-8910" to be "running" Jan 8 06:52:07.036: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 178.348296ms Jan 8 06:52:09.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2.35794258s Jan 8 06:52:11.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4.357423516s Jan 8 06:52:13.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 6.358083829s Jan 8 06:52:15.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 8.357748646s Jan 8 06:52:17.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 10.357048105s Jan 8 06:52:19.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 12.357754002s Jan 8 06:52:21.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 14.357034163s Jan 8 06:52:23.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 16.357356799s Jan 8 06:52:25.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 18.357869691s Jan 8 06:52:27.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 20.357092412s Jan 8 06:52:29.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 22.357397547s Jan 8 06:52:31.217: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 24.358682978s Jan 8 06:52:33.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 26.357183913s Jan 8 06:52:35.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 28.357953467s Jan 8 06:52:37.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 30.357594329s Jan 8 06:52:39.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 32.357842889s Jan 8 06:52:41.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 34.357820645s Jan 8 06:52:43.217: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 36.358442453s Jan 8 06:52:45.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 38.357112423s Jan 8 06:52:47.218: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 40.359895713s Jan 8 06:52:49.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 42.357766192s Jan 8 06:52:51.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 44.35754133s Jan 8 06:52:53.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 46.357690248s Jan 8 06:52:55.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 48.357862946s Jan 8 06:52:57.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 50.357758018s Jan 8 06:52:59.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 52.357770752s Jan 8 06:53:01.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 54.357791547s Jan 8 06:53:03.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 56.358288352s Jan 8 06:53:05.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 58.358025869s Jan 8 06:53:07.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.357424702s Jan 8 06:53:09.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.357387031s Jan 8 06:53:11.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.358216728s Jan 8 06:53:13.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.357440033s Jan 8 06:53:15.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.357203472s Jan 8 06:53:17.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.357417538s Jan 8 06:53:19.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.3578967s Jan 8 06:53:21.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.357434369s Jan 8 06:53:23.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.357035683s Jan 8 06:53:25.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.357391928s Jan 8 06:53:27.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.357905637s Jan 8 06:53:29.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.357733491s Jan 8 06:53:31.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.357831156s Jan 8 06:53:33.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.35722611s Jan 8 06:53:35.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.357424875s Jan 8 06:53:37.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.357943455s Jan 8 06:53:39.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.357923872s Jan 8 06:53:41.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.357328424s Jan 8 06:53:43.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.357165244s Jan 8 06:53:45.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.357259323s Jan 8 06:53:47.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.357093457s Jan 8 06:53:49.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.357891293s Jan 8 06:53:51.218: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.35982222s Jan 8 06:53:53.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.357833534s Jan 8 06:53:55.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.358025702s Jan 8 06:53:57.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.357079178s Jan 8 06:53:59.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.357901991s Jan 8 06:54:01.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.357562185s Jan 8 06:54:03.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.357806561s Jan 8 06:54:05.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.357757706s Jan 8 06:54:07.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.35710664s Jan 8 06:54:09.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.35762659s Jan 8 06:54:11.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.357521504s Jan 8 06:54:13.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.357636667s Jan 8 06:54:15.217: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.358455156s Jan 8 06:54:17.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.357909519s Jan 8 06:54:19.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.357652251s Jan 8 06:54:21.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.356938223s Jan 8 06:54:23.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.356964481s Jan 8 06:54:25.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.357295015s Jan 8 06:54:27.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.357812155s Jan 8 06:54:29.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.356625809s Jan 8 06:54:31.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.356929248s Jan 8 06:54:33.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.357696909s Jan 8 06:54:35.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.357980721s Jan 8 06:54:37.217: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.358628574s Jan 8 06:54:39.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.356935238s Jan 8 06:54:41.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.357459547s Jan 8 06:54:43.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.358238074s Jan 8 06:54:45.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.357119028s Jan 8 06:54:47.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.357989134s Jan 8 06:54:49.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.357591481s Jan 8 06:54:51.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.357625533s Jan 8 06:54:53.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.357950503s Jan 8 06:54:55.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.357782075s Jan 8 06:54:57.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.356952082s Jan 8 06:54:59.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.35704079s Jan 8 06:55:01.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.357817224s Jan 8 06:55:03.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.35721531s Jan 8 06:55:05.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.358083137s Jan 8 06:55:07.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.357169339s Jan 8 06:55:09.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.35720366s Jan 8 06:55:11.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.357921339s Jan 8 06:55:13.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.357661151s Jan 8 06:55:15.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.357081012s Jan 8 06:55:17.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.357861019s Jan 8 06:55:19.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.35752036s Jan 8 06:55:21.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.356945945s Jan 8 06:55:23.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.357174705s Jan 8 06:55:25.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.357779657s Jan 8 06:55:27.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.357885009s Jan 8 06:55:29.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.358347305s Jan 8 06:55:31.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.357880421s Jan 8 06:55:33.217: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.358864879s Jan 8 06:55:35.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.357626528s Jan 8 06:55:37.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.35727853s Jan 8 06:55:39.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.357159375s Jan 8 06:55:41.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.357176792s Jan 8 06:55:43.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.357918364s Jan 8 06:55:45.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.357416971s Jan 8 06:55:47.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.357288842s Jan 8 06:55:49.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.357229667s Jan 8 06:55:51.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.357497981s Jan 8 06:55:53.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.357308117s Jan 8 06:55:55.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.357261242s Jan 8 06:55:57.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.358144023s Jan 8 06:55:59.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.357550018s Jan 8 06:56:01.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.35771925s Jan 8 06:56:03.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.357258124s Jan 8 06:56:05.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.35803211s Jan 8 06:56:07.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.35736751s Jan 8 06:56:09.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.357422613s Jan 8 06:56:11.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.357206786s Jan 8 06:56:13.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.358071109s Jan 8 06:56:15.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.358260812s Jan 8 06:56:17.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.35731919s Jan 8 06:56:19.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.35743405s Jan 8 06:56:21.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.357845599s Jan 8 06:56:23.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.3573881s Jan 8 06:56:25.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.358010814s Jan 8 06:56:27.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.357178606s Jan 8 06:56:29.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.357254279s Jan 8 06:56:31.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.357534306s Jan 8 06:56:33.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.35791561s Jan 8 06:56:35.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.35755471s Jan 8 06:56:37.217: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.359052146s Jan 8 06:56:39.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.35755053s Jan 8 06:56:41.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.357894975s Jan 8 06:56:43.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.358352258s Jan 8 06:56:45.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.357019846s Jan 8 06:56:47.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.357629692s Jan 8 06:56:49.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.357657036s Jan 8 06:56:51.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.357565305s Jan 8 06:56:53.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.357477089s Jan 8 06:56:55.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.357134129s Jan 8 06:56:57.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.357134733s Jan 8 06:56:59.215: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.356499383s Jan 8 06:57:01.217: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.358483626s Jan 8 06:57:03.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.35775456s Jan 8 06:57:05.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.357801899s Jan 8 06:57:07.216: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.35771083s Jan 8 06:57:07.395: INFO: Pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.536464424s Jan 8 06:57:07.396: INFO: Unexpected error: <*pod.timeoutError | 0xc005450390>: { msg: "timed out while waiting for pod topology-8910/pod-937b6812-ddb0-47fe-af8b-07288c742e73 to be running", observedObjects: [ <*v1.Pod | 0xc00295b200>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "pod-937b6812-ddb0-47fe-af8b-07288c742e73", GenerateName: "", Namespace: "topology-8910", SelfLink: "", UID: "f3109315-7294-425a-bfc2-fde1d6f77595", ResourceVersion: "42181", Generation: 0, CreationTimestamp: { Time: { wall: 0, ext: 63808757526, loc: { name: "Local", zone: [ {name: "UTC", offset: 0, isDST: false}, ], tx: [ { when: -576460752303423488, index: 0, isstd: false, isutc: false, }, ], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: "UTC", offset: 0, isDST: false}, }, }, }, DeletionTimestamp: nil, DeletionGracePeriodSeconds: nil, Labels: nil, Annotations: nil, OwnerReferences: nil, Finalizers: nil, ManagedFields: [ { Manager: "e2e.test", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63808757526, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"write-pod\\\"}\":{\".\":{},\"f:command\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:securityContext\":{\".\":{},\"f:privileged\":{}},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{},\"f:volumeMounts\":{\".\":{},\"k:{\\\"mountPath\\\":\\\"/mnt/volume1\\\"}\":{\".\":{},\"f:mountPath\":{},\"f:name\":{}}}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{\".\":{},\"f:fsGroup\":{},\"f:seLinuxOptions\":{\".\":{},\"f:level\":{}}},\"f:terminationGracePeriodSeconds\":{},\"f:volumes\":{\".\":{},\"k:{\\\"name\\\":\\\"volume1\\\"}\":{\".\":{},\"f:name\":{},\"f:persistentVolumeClaim\":{\".\":{},\"f:claimName\":{}}}}}}", }, Subresource: "", }, { Manager: "kubelet", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63808757530, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}", }, Subresource: "status", }, ], }, Spec: { Volumes: [ { Name: "volume1", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: { ClaimName: "pvc-78pz6", ReadOnly: false, }, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: nil, PortworxVolume: nil, ScaleIO: nil, StorageOS: nil, CSI: nil, Ephemeral: nil, }, }, { Name: "kube-api-access-7797f", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Jan 8 06:57:07.396: FAIL: timed out while waiting for pod topology-8910/pod-937b6812-ddb0-47fe-af8b-07288c742e73 to be running Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*topologyTestSuite).DefineTests.func3() test/e2e/storage/testsuites/topology.go:179 +0x4b5 �[1mSTEP:�[0m Deleting pod �[38;5;243m01/08/23 06:57:07.396�[0m Jan 8 06:57:07.396: INFO: Deleting pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73" in namespace "topology-8910" Jan 8 06:57:07.578: INFO: Wait up to 5m0s for pod "pod-937b6812-ddb0-47fe-af8b-07288c742e73" to be fully deleted �[1mSTEP:�[0m Deleting pvc �[38;5;243m01/08/23 06:57:09.935�[0m Jan 8 06:57:10.295: INFO: Deleting PersistentVolumeClaim "pvc-78pz6" Jan 8 06:57:10.480: INFO: Waiting up to 5m0s for PersistentVolume pvc-dee24227-890e-48fe-b869-a255c7997280 to get deleted Jan 8 06:57:10.658: INFO: PersistentVolume pvc-dee24227-890e-48fe-b869-a255c7997280 found and phase=Released (178.163506ms) Jan 8 06:57:15.838: INFO: PersistentVolume pvc-dee24227-890e-48fe-b869-a255c7997280 found and phase=Released (5.357803192s) Jan 8 06:57:21.017: INFO: PersistentVolume pvc-dee24227-890e-48fe-b869-a255c7997280 found and phase=Released (10.536531302s) Jan 8 06:57:26.195: INFO: PersistentVolume pvc-dee24227-890e-48fe-b869-a255c7997280 was removed �[1mSTEP:�[0m Deleting sc �[38;5;243m01/08/23 06:57:26.195�[0m Jan 8 06:57:26.376: INFO: Unexpected error: while cleaning up driver: <*pod.timeoutError | 0xc005450390>: { msg: "timed out while waiting for pod topology-8910/pod-937b6812-ddb0-47fe-af8b-07288c742e73 to be running", observedObjects: [ <*v1.Pod | 0xc00295b200>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "pod-937b6812-ddb0-47fe-af8b-07288c742e73", GenerateName: "", Namespace: "topology-8910", SelfLink: "", UID: "f3109315-7294-425a-bfc2-fde1d6f77595", ResourceVersion: "42181", Generation: 0, CreationTimestamp: { Time: { wall: 0, ext: 63808757526, loc: { name: "Local", zone: [ {name: "UTC", offset: 0, isDST: false}, ], tx: [ { when: -576460752303423488, index: 0, isstd: false, isutc: false, }, ], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: "UTC", offset: 0, isDST: false}, }, }, }, DeletionTimestamp: nil, DeletionGracePeriodSeconds: nil, Labels: nil, Annotations: nil, OwnerReferences: nil, Finalizers: nil, ManagedFields: [ { Manager: "e2e.test", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63808757526, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"write-pod\\\"}\":{\".\":{},\"f:command\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:securityContext\":{\".\":{},\"f:privileged\":{}},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{},\"f:volumeMounts\":{\".\":{},\"k:{\\\"mountPath\\\":\\\"/mnt/volume1\\\"}\":{\".\":{},\"f:mountPath\":{},\"f:name\":{}}}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{\".\":{},\"f:fsGroup\":{},\"f:seLinuxOptions\":{\".\":{},\"f:level\":{}}},\"f:terminationGracePeriodSeconds\":{},\"f:volumes\":{\".\":{},\"k:{\\\"name\\\":\\\"volume1\\\"}\":{\".\":{},\"f:name\":{},\"f:persistentVolumeClaim\":{\".\":{},\"f:claimName\":{}}}}}}", }, Subresource: "", }, { Manager: "kubelet", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63808757530, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}", }, Subresource: "status", }, ], }, Spec: { Volumes: [ { Name: "volume1", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: { ClaimName: "pvc-78pz6", ReadOnly: false, }, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: nil, PortworxVolume: nil, ScaleIO: nil, StorageOS: nil, CSI: nil, Ephemeral: nil, }, }, { Name: "kube-api-access-7797f", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Jan 8 06:57:26.376: FAIL: while cleaning up driver: timed out while waiting for pod topology-8910/pod-937b6812-ddb0-47fe-af8b-07288c742e73 to be running Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*topologyTestSuite).DefineTests.func2({0xc004ce3320, 0xc0012cc780, {0xc004ce3320, {{0x7636363, 0x1c}, {0x0, 0x0}, {0x75c5249, 0x9}, {0x0, ...}, ...}, ...}, ...}) test/e2e/storage/testsuites/topology.go:158 +0x8e k8s.io/kubernetes/test/e2e/storage/testsuites.(*topologyTestSuite).DefineTests.func3.1() test/e2e/storage/testsuites/topology.go:166 +0x55 panic({0x70eb680, 0xc000cd8ee0}) /usr/local/go/src/runtime/panic.go:884 +0x212 k8s.io/kubernetes/test/e2e/framework.Fail({0xc003d38000, 0x64}, {0xc0008abbc8?, 0xc003d38000?, 0xc0008abbf0?}) test/e2e/framework/log.go:61 +0x145 k8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0x1, {0x7fa44e0, 0xc005450390}, {0x0?, 0xc004b76f13?, 0x0?}) test/e2e/framework/expect.go:76 +0x267 k8s.io/kubernetes/test/e2e/framework.ExpectNoError(...) test/e2e/framework/expect.go:43 k8s.io/kubernetes/test/e2e/storage/testsuites.(*topologyTestSuite).DefineTests.func3() test/e2e/storage/testsuites/topology.go:179 +0x4b5 [AfterEach] [Testpattern: Dynamic PV (delayed binding)] topology test/e2e/framework/node/init/init.go:32 Jan 8 06:57:26.376: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [Testpattern: Dynamic PV (delayed binding)] topology test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [Testpattern: Dynamic PV (delayed binding)] topology dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/08/23 06:57:26.556�[0m �[1mSTEP:�[0m Collecting events from namespace "topology-8910". �[38;5;243m01/08/23 06:57:26.556�[0m �[1mSTEP:�[0m Found 16 events. �[38;5;243m01/08/23 06:57:26.734�[0m Jan 8 06:57:26.735: INFO: At 2023-01-08 06:52:06 +0000 UTC - event for pvc-78pz6: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding Jan 8 06:57:26.735: INFO: At 2023-01-08 06:52:06 +0000 UTC - event for pvc-78pz6: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-vzzj9_eb492b31-b783-48e6-ab2b-bc22df496b4f } Provisioning: External provisioner is provisioning volume for claim "topology-8910/pvc-78pz6" Jan 8 06:57:26.735: INFO: At 2023-01-08 06:52:06 +0000 UTC - event for pvc-78pz6: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator Jan 8 06:57:26.735: INFO: At 2023-01-08 06:52:10 +0000 UTC - event for pod-937b6812-ddb0-47fe-af8b-07288c742e73: {default-scheduler } Scheduled: Successfully assigned topology-8910/pod-937b6812-ddb0-47fe-af8b-07288c742e73 to i-0685adc783a822be6 Jan 8 06:57:26.735: INFO: At 2023-01-08 06:52:10 +0000 UTC - event for pvc-78pz6: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-vzzj9_eb492b31-b783-48e6-ab2b-bc22df496b4f } ProvisioningSucceeded: Successfully provisioned volume pvc-dee24227-890e-48fe-b869-a255c7997280 Jan 8 06:57:26.735: INFO: At 2023-01-08 06:52:13 +0000 UTC - event for pod-937b6812-ddb0-47fe-af8b-07288c742e73: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-dee24227-890e-48fe-b869-a255c7997280" Jan 8 06:57:26.735: INFO: At 2023-01-08 06:52:13 +0000 UTC - event for pod-937b6812-ddb0-47fe-af8b-07288c742e73: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5a5b474f0f853445ec06dfb609ec087f6f0d9a0928d5f48af1b93b59708a9bce": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:57:26.735: INFO: At 2023-01-08 06:52:27 +0000 UTC - event for pod-937b6812-ddb0-47fe-af8b-07288c742e73: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5d38b803dfb80a99e083396ec9858bd9e18cfad9a7e028ec2ee68cb79c50d919": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:57:26.735: INFO: At 2023-01-08 06:52:39 +0000 UTC - event for pod-937b6812-ddb0-47fe-af8b-07288c742e73: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "27b851fefeb4fbf7c06ae113ea2d99ca88ea41b64a249a9eefa086a3fc08cdd4": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:57:26.735: INFO: At 2023-01-08 06:52:54 +0000 UTC - event for pod-937b6812-ddb0-47fe-af8b-07288c742e73: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e7762e865999be01950445701f6dcdb7e334c53a8a84d9c2ebdc8d9530da53ef": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:57:26.735: INFO: At 2023-01-08 06:53:07 +0000 UTC - event for pod-937b6812-ddb0-47fe-af8b-07288c742e73: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a4ce8a6e45bc50363a6ac0cce11e006da20b0cdb93410bc996ff6497b097ca04": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:57:26.735: INFO: At 2023-01-08 06:53:20 +0000 UTC - event for pod-937b6812-ddb0-47fe-af8b-07288c742e73: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "71f0ba71d81009eb8c378ad6f65fc35efb1479fc09468286e67ef11b036b1812": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:57:26.735: INFO: At 2023-01-08 06:53:34 +0000 UTC - event for pod-937b6812-ddb0-47fe-af8b-07288c742e73: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f53b55db4dceadc0d1f700f626e1b7a36213700eb2068d5057c4eb34655ecef7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:57:26.735: INFO: At 2023-01-08 06:53:45 +0000 UTC - event for pod-937b6812-ddb0-47fe-af8b-07288c742e73: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c29e006e9d0942e8f5eef6612c025f0973cf25f1979524f83bfaf5391a2bebed": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:57:26.735: INFO: At 2023-01-08 06:53:59 +0000 UTC - event for pod-937b6812-ddb0-47fe-af8b-07288c742e73: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8d0467c28524818f7a2f006370fbe2db73611e215abde5ef852b3be9e0f95969": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:57:26.735: INFO: At 2023-01-08 06:54:15 +0000 UTC - event for pod-937b6812-ddb0-47fe-af8b-07288c742e73: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0fb258c6984808106a8b0cf25ddc5632e583d1fc1d00600699f0c6bec2649d6c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:57:26.912: INFO: POD NODE PHASE GRACE CONDITIONS Jan 8 06:57:26.913: INFO: Jan 8 06:57:27.092: INFO: Logging node info for node i-0685adc783a822be6 Jan 8 06:57:27.270: INFO: Node Info: &Node{ObjectMeta:{i-0685adc783a822be6 3eb588c7-cc76-44b4-b28a-492b9989254b 44173 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0685adc783a822be6 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0685adc783a822be6 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.112 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0685adc783a822be6"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:57:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-08 06:57:18 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0685adc783a822be6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:17 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:17 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:17 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:57:17 +0000 UTC,LastTransitionTime:2023-01-08 06:25:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.112,},NodeAddress{Type:ExternalIP,Address:43.201.46.149,},NodeAddress{Type:InternalDNS,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-201-46-149.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dccafbf198926b990228825ae0608,SystemUUID:ec2dccaf-bf19-8926-b990-228825ae0608,BootID:b8ff627c-af94-4730-be1d-6c7bfbfc818f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0488b0c43122e11c1 kubernetes.io/csi/ebs.csi.aws.com^vol-0ff9b2b6e0e0d5c29],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0488b0c43122e11c1,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0ff9b2b6e0e0d5c29,DevicePath:,},},Config:nil,},} Jan 8 06:57:27.270: INFO: Logging kubelet events for node i-0685adc783a822be6 Jan 8 06:57:27.450: INFO: Logging pods the kubelet thinks is on node i-0685adc783a822be6 Jan 8 06:57:27.638: INFO: inline-volume-tester-9qdkl started at 2023-01-08 06:52:11 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 8 06:57:27.638: INFO: cilium-9vhbx started at 2023-01-08 06:25:38 +0000 UTC (1+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:57:27.638: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:57:27.638: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:56:47 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container csi-attacher ready: false, restart count 0 Jan 8 06:57:27.638: INFO: pod-disruption-failure-ignore-1-kfqqr started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container c ready: false, restart count 0 Jan 8 06:57:27.638: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-gq5pr started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:27.638: INFO: pod-8cb6de86-0572-49a0-a173-3eb48174b2d3 started at 2023-01-08 06:56:59 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container write-pod ready: false, restart count 0 Jan 8 06:57:27.638: INFO: service-proxy-toggled-dh67n started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container service-proxy-toggled ready: false, restart count 0 Jan 8 06:57:27.638: INFO: liveness-51325787-aaa2-4ac1-a3f9-8ff47e3fa793 started at 2023-01-08 06:54:56 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container agnhost-container ready: false, restart count 0 Jan 8 06:57:27.638: INFO: pod-subpath-test-inlinevolume-plpm started at 2023-01-08 06:52:37 +0000 UTC (2+2 container statuses recorded) Jan 8 06:57:27.638: INFO: Init container init-volume-inlinevolume-plpm ready: false, restart count 0 Jan 8 06:57:27.638: INFO: Init container test-init-subpath-inlinevolume-plpm ready: false, restart count 0 Jan 8 06:57:27.638: INFO: Container test-container-subpath-inlinevolume-plpm ready: false, restart count 0 Jan 8 06:57:27.638: INFO: Container test-container-volume-inlinevolume-plpm ready: false, restart count 0 Jan 8 06:57:27.638: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-svbp2 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:27.638: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mpw58 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:27.638: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rrmzm started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:27.638: INFO: csi-mockplugin-0 started at 2023-01-08 06:56:47 +0000 UTC (0+3 container statuses recorded) Jan 8 06:57:27.638: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 06:57:27.638: INFO: Container driver-registrar ready: false, restart count 0 Jan 8 06:57:27.638: INFO: Container mock ready: false, restart count 0 Jan 8 06:57:27.638: INFO: pod-disruption-failure-ignore-0-w4rvn started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container c ready: false, restart count 0 Jan 8 06:57:27.638: INFO: pod-projected-secrets-907df958-d79f-42f9-87ee-41301a6f40be started at 2023-01-08 06:55:13 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container secret-volume-test ready: false, restart count 0 Jan 8 06:57:27.638: INFO: ebs-csi-node-ft7bw started at 2023-01-08 06:25:38 +0000 UTC (0+3 container statuses recorded) Jan 8 06:57:27.638: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:57:27.638: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:57:27.638: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:57:27.638: INFO: suspend-false-to-true-97wd6 started at 2023-01-08 06:53:18 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container c ready: true, restart count 0 Jan 8 06:57:27.638: INFO: csi-mockplugin-0 started at 2023-01-08 06:56:04 +0000 UTC (0+4 container statuses recorded) Jan 8 06:57:27.638: INFO: Container busybox ready: false, restart count 0 Jan 8 06:57:27.638: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 06:57:27.638: INFO: Container driver-registrar ready: false, restart count 0 Jan 8 06:57:27.638: INFO: Container mock ready: false, restart count 0 Jan 8 06:57:27.638: INFO: hostexec-i-0685adc783a822be6-ms669 started at 2023-01-08 06:56:52 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container agnhost-container ready: true, restart count 0 Jan 8 06:57:27.638: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-q97zd started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:27.638: INFO: affinity-clusterip-zxklh started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container affinity-clusterip ready: false, restart count 0 Jan 8 06:57:27.638: INFO: service-proxy-disabled-6pp8x started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container service-proxy-disabled ready: true, restart count 0 Jan 8 06:57:27.638: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2bn4h started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:27.638: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v9tjx started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:27.638: INFO: pod-subpath-test-dynamicpv-28vt started at 2023-01-08 06:57:14 +0000 UTC (0+2 container statuses recorded) Jan 8 06:57:27.638: INFO: Container test-container-subpath-dynamicpv-28vt ready: false, restart count 0 Jan 8 06:57:27.638: INFO: Container test-container-volume-dynamicpv-28vt ready: false, restart count 0 Jan 8 06:57:27.638: INFO: agnhost-primary-9vrbd started at 2023-01-08 06:54:54 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container agnhost-primary ready: false, restart count 0 Jan 8 06:57:27.638: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-97qp6 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:27.638: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-4d46c started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:27.638: INFO: csi-mockplugin-resizer-0 started at 2023-01-08 06:56:47 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container csi-resizer ready: false, restart count 0 Jan 8 06:57:27.638: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-zgmck started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:27.638: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:28.283: INFO: Latency metrics for node i-0685adc783a822be6 Jan 8 06:57:28.283: INFO: Logging node info for node i-0bd196cab651da520 Jan 8 06:57:28.462: INFO: Node Info: &Node{ObjectMeta:{i-0bd196cab651da520 f7f00780-d24e-4c8e-b54d-373598e1d1d2 44022 0 2023-01-08 06:26:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0bd196cab651da520 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0bd196cab651da520 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.104 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bd196cab651da520"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:26:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:57:01 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-08 06:57:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0bd196cab651da520,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:26:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.104,},NodeAddress{Type:ExternalIP,Address:54.180.1.55,},NodeAddress{Type:InternalDNS,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-180-1-55.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec218f9d32a84b2feeb6301255fd576c,SystemUUID:ec218f9d-32a8-4b2f-eeb6-301255fd576c,BootID:accdf7fd-1195-4765-b28e-29d44bf47bac,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-04598dd4ddfa6789b kubernetes.io/csi/ebs.csi.aws.com^vol-06b18f9cd5a55aaf5 kubernetes.io/csi/ebs.csi.aws.com^vol-07a4e46ecc4ccd59e],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-07a4e46ecc4ccd59e,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-04598dd4ddfa6789b,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-06b18f9cd5a55aaf5,DevicePath:,},},Config:nil,},} Jan 8 06:57:28.462: INFO: Logging kubelet events for node i-0bd196cab651da520 Jan 8 06:57:28.643: INFO: Logging pods the kubelet thinks is on node i-0bd196cab651da520 Jan 8 06:57:28.828: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v44bk started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:28.828: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-75r8w started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:28.828: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9mp69 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:28.828: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-cddx5 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:28.828: INFO: pod6b22b1b4-d027-4092-9a22-c1f06499be00 started at 2023-01-08 06:56:02 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container nginx ready: false, restart count 0 Jan 8 06:57:28.828: INFO: pod-subpath-test-dynamicpv-cx9n started at 2023-01-08 06:56:59 +0000 UTC (2+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Init container init-volume-dynamicpv-cx9n ready: false, restart count 0 Jan 8 06:57:28.828: INFO: Init container test-init-subpath-dynamicpv-cx9n ready: false, restart count 0 Jan 8 06:57:28.828: INFO: Container test-container-subpath-dynamicpv-cx9n ready: false, restart count 0 Jan 8 06:57:28.828: INFO: pod-projected-configmaps-143d6d2e-7b9b-47e8-97c7-c2b8adc28191 started at 2023-01-08 06:52:29 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container agnhost-container ready: false, restart count 0 Jan 8 06:57:28.828: INFO: suspend-false-to-true-xwpqd started at 2023-01-08 06:53:18 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container c ready: false, restart count 0 Jan 8 06:57:28.828: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-6df8l started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:28.828: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-vrcjq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:28.828: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9776c started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:28.828: INFO: busybox-scheduling-0abca61c-e823-4136-8618-ce7deb358c0b started at 2023-01-08 06:55:28 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container busybox-scheduling-0abca61c-e823-4136-8618-ce7deb358c0b ready: false, restart count 0 Jan 8 06:57:28.828: INFO: inline-volume-tester-m9vk5 started at 2023-01-08 06:56:50 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 8 06:57:28.828: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9wskq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:28.828: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mphgx started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:28.828: INFO: pod1 started at 2023-01-08 06:51:10 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container agnhost-container ready: true, restart count 0 Jan 8 06:57:28.828: INFO: affinity-clusterip-mrptn started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container affinity-clusterip ready: true, restart count 0 Jan 8 06:57:28.828: INFO: cilium-s6shx started at 2023-01-08 06:26:02 +0000 UTC (1+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:57:28.828: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:57:28.828: INFO: ebs-csi-node-6grlm started at 2023-01-08 06:26:02 +0000 UTC (0+3 container statuses recorded) Jan 8 06:57:28.828: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:57:28.828: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:57:28.828: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:57:28.828: INFO: execpodgmcm4 started at 2023-01-08 06:55:35 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container agnhost-container ready: false, restart count 0 Jan 8 06:57:28.828: INFO: service-proxy-toggled-8nfkz started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container service-proxy-toggled ready: false, restart count 0 Jan 8 06:57:28.828: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-76nxw started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:28.828: INFO: sample-webhook-deployment-865554f4d9-4kppl started at 2023-01-08 06:55:41 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:28.828: INFO: Container sample-webhook ready: false, restart count 0 Jan 8 06:57:29.467: INFO: Latency metrics for node i-0bd196cab651da520 Jan 8 06:57:29.467: INFO: Logging node info for node i-0d149644e2edcf9cf Jan 8 06:57:29.646: INFO: Node Info: &Node{ObjectMeta:{i-0d149644e2edcf9cf b448ef72-6e61-4ad5-aa29-d63cc0f94945 43645 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0d149644e2edcf9cf kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0d149644e2edcf9cf topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.115 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0d149644e2edcf9cf"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:56:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0d149644e2edcf9cf,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.115,},NodeAddress{Type:ExternalIP,Address:3.35.219.218,},NodeAddress{Type:InternalDNS,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-35-219-218.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec26c6a9af66729e278545162595eb2d,SystemUUID:ec26c6a9-af66-729e-2785-45162595eb2d,BootID:959da9e3-fb86-442a-958f-2a49c509cdf4,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 8 06:57:29.646: INFO: Logging kubelet events for node i-0d149644e2edcf9cf Jan 8 06:57:29.827: INFO: Logging pods the kubelet thinks is on node i-0d149644e2edcf9cf Jan 8 06:57:30.013: INFO: csi-mockplugin-0 started at 2023-01-08 06:52:13 +0000 UTC (0+3 container statuses recorded) Jan 8 06:57:30.013: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 06:57:30.013: INFO: Container driver-registrar ready: false, restart count 0 Jan 8 06:57:30.013: INFO: Container mock ready: false, restart count 0 Jan 8 06:57:30.013: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-njh2f started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:30.013: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:30.013: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2vh6v started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:30.013: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:30.013: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nt9jr started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:30.013: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0 Jan 8 06:57:30.013: INFO: cilium-58twl started at 2023-01-08 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 8 06:57:30.013: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:57:30.013: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:57:30.013: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rv5qq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:30.013: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:30.013: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-flhps started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:30.013: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:30.013: INFO: ebs-csi-node-lfcwb started at 2023-01-08 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 8 06:57:30.013: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:57:30.013: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:57:30.013: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:57:30.013: INFO: coredns-559769c974-9q4qg started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:30.013: INFO: Container coredns ready: true, restart count 0 Jan 8 06:57:30.013: INFO: csi-hostpathplugin-0 started at 2023-01-08 06:52:05 +0000 UTC (0+7 container statuses recorded) Jan 8 06:57:30.013: INFO: Container csi-attacher ready: false, restart count 0 Jan 8 06:57:30.013: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 06:57:30.013: INFO: Container csi-resizer ready: false, restart count 0 Jan 8 06:57:30.013: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 8 06:57:30.013: INFO: Container hostpath ready: false, restart count 0 Jan 8 06:57:30.013: INFO: Container liveness-probe ready: false, restart count 0 Jan 8 06:57:30.013: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 8 06:57:30.013: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-xrmq4 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:30.013: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:30.013: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-7gwsq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:30.013: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:30.013: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nsfnm started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:30.013: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:30.013: INFO: coredns-autoscaler-7cb5c5b969-xqnk5 started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:30.013: INFO: Container autoscaler ready: true, restart count 0 Jan 8 06:57:30.013: INFO: service-proxy-disabled-v5f82 started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:30.013: INFO: Container service-proxy-disabled ready: true, restart count 0 Jan 8 06:57:30.013: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-tgn6r started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:30.013: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:30.013: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:52:13 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:30.013: INFO: Container csi-attacher ready: false, restart count 0 Jan 8 06:57:30.013: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9b2xq started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:30.013: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:30.755: INFO: Latency metrics for node i-0d149644e2edcf9cf Jan 8 06:57:30.755: INFO: Logging node info for node i-0eee6113fd9f81d94 Jan 8 06:57:30.934: INFO: Node Info: &Node{ObjectMeta:{i-0eee6113fd9f81d94 6de592fc-f11c-4ba3-afa0-707bb4ca29b2 44024 0 2023-01-08 06:23:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0eee6113fd9f81d94 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.48.64 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eee6113fd9f81d94"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-08 06:23:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-08 06:24:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-08 06:24:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-08 06:57:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0eee6113fd9f81d94,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:24:35 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.48.64,},NodeAddress{Type:ExternalIP,Address:3.35.206.95,},NodeAddress{Type:InternalDNS,Address:i-0eee6113fd9f81d94.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0eee6113fd9f81d94.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-35-206-95.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d8cf21861b15bb593210ff59f6044,SystemUUID:ec2d8cf2-1861-b15b-b593-210ff59f6044,BootID:fcad947c-a63f-4722-9a63-22eac74bea9f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43453935,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42803460,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967354,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 8 06:57:30.934: INFO: Logging kubelet events for node i-0eee6113fd9f81d94 Jan 8 06:57:31.115: INFO: Logging pods the kubelet thinks is on node i-0eee6113fd9f81d94 Jan 8 06:57:31.299: INFO: etcd-manager-events-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:31.299: INFO: Container etcd-manager ready: true, restart count 0 Jan 8 06:57:31.299: INFO: etcd-manager-main-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:31.299: INFO: Container etcd-manager ready: true, restart count 0 Jan 8 06:57:31.299: INFO: kops-controller-242qq started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:31.299: INFO: Container kops-controller ready: true, restart count 0 Jan 8 06:57:31.299: INFO: cilium-f22px started at 2023-01-08 06:24:21 +0000 UTC (1+1 container statuses recorded) Jan 8 06:57:31.299: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:57:31.299: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:57:31.299: INFO: dns-controller-69987775c6-v4cmg started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:31.299: INFO: Container dns-controller ready: true, restart count 0 Jan 8 06:57:31.299: INFO: kube-scheduler-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:31.299: INFO: Container kube-scheduler ready: true, restart count 0 Jan 8 06:57:31.299: INFO: kube-apiserver-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+2 container statuses recorded) Jan 8 06:57:31.299: INFO: Container healthcheck ready: true, restart count 0 Jan 8 06:57:31.299: INFO: Container kube-apiserver ready: true, restart count 1 Jan 8 06:57:31.299: INFO: kube-controller-manager-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:31.299: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 8 06:57:31.299: INFO: ebs-csi-node-nspqr started at 2023-01-08 06:24:21 +0000 UTC (0+3 container statuses recorded) Jan 8 06:57:31.299: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:57:31.299: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:57:31.299: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:57:31.299: INFO: aws-cloud-controller-manager-d8d56 started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:31.299: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 8 06:57:31.299: INFO: cilium-operator-5dd44dc49f-fd8vl started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:31.299: INFO: Container cilium-operator ready: true, restart count 0 Jan 8 06:57:31.299: INFO: ebs-csi-controller-89b649c94-vzzj9 started at 2023-01-08 06:24:21 +0000 UTC (0+5 container statuses recorded) Jan 8 06:57:31.299: INFO: Container csi-attacher ready: true, restart count 0 Jan 8 06:57:31.299: INFO: Container csi-provisioner ready: true, restart count 0 Jan 8 06:57:31.299: INFO: Container csi-resizer ready: true, restart count 0 Jan 8 06:57:31.299: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:57:31.299: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:57:31.890: INFO: Latency metrics for node i-0eee6113fd9f81d94 Jan 8 06:57:31.890: INFO: Logging node info for node i-0f469f08be563dad4 Jan 8 06:57:32.068: INFO: Node Info: &Node{ObjectMeta:{i-0f469f08be563dad4 fddb222c-a6ac-415f-83dc-4b71acb0e5ab 43138 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0f469f08be563dad4 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0f469f08be563dad4 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.57.173 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0f469f08be563dad4"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:55:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0f469f08be563dad4,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.57.173,},NodeAddress{Type:ExternalIP,Address:3.38.152.241,},NodeAddress{Type:InternalDNS,Address:i-0f469f08be563dad4.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f469f08be563dad4.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-38-152-241.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2abc7b1d816532cfb4787d85cb98ea,SystemUUID:ec2abc7b-1d81-6532-cfb4-787d85cb98ea,BootID:442daa6f-46d1-4c32-9182-8b6204c22aba,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 8 06:57:32.068: INFO: Logging kubelet events for node i-0f469f08be563dad4 Jan 8 06:57:32.249: INFO: Logging pods the kubelet thinks is on node i-0f469f08be563dad4 Jan 8 06:57:32.433: INFO: cilium-24rbm started at 2023-01-08 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 8 06:57:32.433: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:57:32.433: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:57:32.433: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-ktkbb started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:32.433: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:32.433: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-dvpgx started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:32.433: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:32.433: INFO: ebs-csi-node-9s8qc started at 2023-01-08 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 8 06:57:32.433: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:57:32.433: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:57:32.433: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:57:32.433: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-x88cb started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:32.433: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:32.433: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-h5c5c started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:32.433: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:32.433: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-jc58p started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:32.433: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:32.433: INFO: service-proxy-disabled-bqbxv started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:32.433: INFO: Container service-proxy-disabled ready: true, restart count 0 Jan 8 06:57:32.433: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-gdlbp started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:32.433: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:32.433: INFO: service-proxy-toggled-7qcjj started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:32.433: INFO: Container service-proxy-toggled ready: true, restart count 0 Jan 8 06:57:32.433: INFO: coredns-559769c974-f9s4x started at 2023-01-08 06:26:19 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:32.433: INFO: Container coredns ready: true, restart count 0 Jan 8 06:57:32.433: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-glvzd started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:32.433: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:32.433: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-cwg4t started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:32.433: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:32.433: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-57rwf started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:32.433: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0 Jan 8 06:57:32.433: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-b6m8x started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:32.433: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:57:32.433: INFO: affinity-clusterip-8kclb started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded) Jan 8 06:57:32.433: INFO: Container affinity-clusterip ready: false, restart count 0 Jan 8 06:57:33.140: INFO: Latency metrics for node i-0f469f08be563dad4 [DeferCleanup (Each)] [Testpattern: Dynamic PV (delayed binding)] topology tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "topology-8910" for this suite. �[38;5;243m01/08/23 06:57:33.14�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-apps\]\sStatefulSet\sBasic\sStatefulSet\sfunctionality\s\[StatefulSetBasic\]\sshould\sprovide\sbasic\sidentity$'
test/e2e/framework/statefulset/wait.go:58
k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801e128?, 0xc003e7e680}, 0x3, 0x3, 0xc002a03900)
test/e2e/framework/statefulset/wait.go:58 +0xf9
k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
test/e2e/framework/statefulset/wait.go:80
k8s.io/kubernetes/test/e2e/apps.glob..func10.2.3()
test/e2e/apps/statefulset.go:161 +0x41e
[BeforeEach] [sig-apps] StatefulSet
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/08/23 06:42:47.085�[0m
Jan 8 06:42:47.085: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename statefulset �[38;5;243m01/08/23 06:42:47.086�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/08/23 06:42:47.617�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/08/23 06:42:47.969�[0m
[BeforeEach] [sig-apps] StatefulSet
test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-apps] StatefulSet
test/e2e/apps/statefulset.go:98
[BeforeEach] Basic StatefulSet functionality [StatefulSetBasic]
test/e2e/apps/statefulset.go:113
�[1mSTEP:�[0m Creating service test in namespace statefulset-4697 �[38;5;243m01/08/23 06:42:48.321�[0m
[It] should provide basic identity
test/e2e/apps/statefulset.go:134
�[1mSTEP:�[0m Creating statefulset ss in namespace statefulset-4697 �[38;5;243m01/08/23 06:42:48.5�[0m
Jan 8 06:42:48.684: INFO: Default storage class: "kops-csi-1-21"
�[1mSTEP:�[0m Saturating stateful set ss �[38;5;243m01/08/23 06:42:48.864�[0m
Jan 8 06:42:48.864: INFO: Waiting for stateful pod at index 0 to enter Running
Jan 8 06:42:49.041: INFO: Waiting for pod ss-0 to enter Running - Ready=false, currently Pending - Ready=false
Jan 8 06:42:59.219: INFO: Waiting for pod ss-0 to enter Running - Ready=false, currently Pending - Ready=false
Jan 8 06:43:09.218: INFO: Waiting for pod ss-0 to enter Running - Ready=false, currently Running - Ready=false
Jan 8 06:43:09.218: INFO: Resuming stateful pod at index 0
Jan 8 06:43:09.395: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-0 -- /bin/sh -x -c dd if=/dev/zero of=/data/statefulset-continue bs=1 count=1 conv=fsync'
Jan 8 06:43:11.221: INFO: stderr: "+ dd 'if=/dev/zero' 'of=/data/statefulset-continue' 'bs=1' 'count=1' 'conv=fsync'\n1+0 records in\n1+0 records out\n"
Jan 8 06:43:11.221: INFO: stdout: ""
Jan 8 06:43:11.221: INFO: Resumed pod ss-0
Jan 8 06:43:11.221: INFO: Waiting for stateful pod at index 1 to enter Running
Jan 8 06:43:11.398: INFO: Found 1 stateful pods, waiting for 2
Jan 8 06:43:21.578: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=true
Jan 8 06:43:21.578: INFO: Waiting for pod ss-1 to enter Running - Ready=false, currently Pending - Ready=false
Jan 8 06:43:31.578: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=true
Jan 8 06:43:31.578: INFO: Waiting for pod ss-1 to enter Running - Ready=false, currently Pending - Ready=false
Jan 8 06:43:41.576: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=true
Jan 8 06:43:41.576: INFO: Waiting for pod ss-1 to enter Running - Ready=false, currently Running - Ready=false
Jan 8 06:43:41.576: INFO: Resuming stateful pod at index 1
Jan 8 06:43:41.753: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-1 -- /bin/sh -x -c dd if=/dev/zero of=/data/statefulset-continue bs=1 count=1 conv=fsync'
Jan 8 06:43:43.504: INFO: stderr: "+ dd 'if=/dev/zero' 'of=/data/statefulset-continue' 'bs=1' 'count=1' 'conv=fsync'\n1+0 records in\n1+0 records out\n"
Jan 8 06:43:43.504: INFO: stdout: ""
Jan 8 06:43:43.504: INFO: Resumed pod ss-1
Jan 8 06:43:43.504: INFO: Waiting for stateful pod at index 2 to enter Running
Jan 8 06:43:43.684: INFO: Found 2 stateful pods, waiting for 3
Jan 8 06:43:53.863: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=true
Jan 8 06:43:53.863: INFO: Waiting for pod ss-1 to enter Running - Ready=true, currently Running - Ready=true
Jan 8 06:43:53.863: INFO: Waiting for pod ss-2 to enter Running - Ready=false, currently Pending - Ready=false
Jan 8 06:44:03.863: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=true
Jan 8 06:44:03.863: INFO: Waiting for pod ss-1 to enter Running - Ready=true, currently Running - Ready=true
Jan 8 06:44:03.863: INFO: Waiting for pod ss-2 to enter Running - Ready=false, currently Pending - Ready=false
Jan 8 06:44:13.862: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=true
Jan 8 06:44:13.862: INFO: Waiting for pod ss-1 to enter Running - Ready=true, currently Running - Ready=true
Jan 8 06:44:13.862: INFO: Waiting for pod ss-2 to enter Running - Ready=false, currently Running - Ready=false
Jan 8 06:44:13.862: INFO: Resuming stateful pod at index 2
Jan 8 06:44:14.039: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-2 -- /bin/sh -x -c dd if=/dev/zero of=/data/statefulset-continue bs=1 count=1 conv=fsync'
Jan 8 06:44:15.824: INFO: stderr: "+ dd 'if=/dev/zero' 'of=/data/statefulset-continue' 'bs=1' 'count=1' 'conv=fsync'\n1+0 records in\n1+0 records out\n"
Jan 8 06:44:15.824: INFO: stdout: ""
Jan 8 06:44:15.824: INFO: Resumed pod ss-2
�[1mSTEP:�[0m Verifying statefulset mounted data directory is usable �[38;5;243m01/08/23 06:44:15.824�[0m
Jan 8 06:44:16.005: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-0 -- /bin/sh -x -c ls -idlh /data'
Jan 8 06:44:17.777: INFO: stderr: "+ ls -idlh /data\n"
Jan 8 06:44:17.777: INFO: stdout: " 2 drwxr-xr-x 3 root root 4.0K Jan 8 06:43 /data\n"
Jan 8 06:44:17.777: INFO: stdout of ls -idlh /data on ss-0: 2 drwxr-xr-x 3 root root 4.0K Jan 8 06:43 /data
Jan 8 06:44:17.777: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-1 -- /bin/sh -x -c ls -idlh /data'
Jan 8 06:44:19.629: INFO: stderr: "+ ls -idlh /data\n"
Jan 8 06:44:19.629: INFO: stdout: " 2 drwxr-xr-x 3 root root 4.0K Jan 8 06:43 /data\n"
Jan 8 06:44:19.629: INFO: stdout of ls -idlh /data on ss-1: 2 drwxr-xr-x 3 root root 4.0K Jan 8 06:43 /data
Jan 8 06:44:19.629: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-2 -- /bin/sh -x -c ls -idlh /data'
Jan 8 06:44:21.455: INFO: stderr: "+ ls -idlh /data\n"
Jan 8 06:44:21.455: INFO: stdout: " 2 drwxr-xr-x 3 root root 4.0K Jan 8 06:44 /data\n"
Jan 8 06:44:21.455: INFO: stdout of ls -idlh /data on ss-2: 2 drwxr-xr-x 3 root root 4.0K Jan 8 06:44 /data
Jan 8 06:44:21.633: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-0 -- /bin/sh -x -c find /data'
Jan 8 06:44:23.425: INFO: stderr: "+ find /data\n"
Jan 8 06:44:23.425: INFO: stdout: "/data\n/data/statefulset-continue\n/data/lost+found\n"
Jan 8 06:44:23.425: INFO: stdout of find /data on ss-0: /data
/data/statefulset-continue
/data/lost+found
Jan 8 06:44:23.426: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-1 -- /bin/sh -x -c find /data'
Jan 8 06:44:25.192: INFO: stderr: "+ find /data\n"
Jan 8 06:44:25.192: INFO: stdout: "/data\n/data/lost+found\n/data/statefulset-continue\n"
Jan 8 06:44:25.192: INFO: stdout of find /data on ss-1: /data
/data/lost+found
/data/statefulset-continue
Jan 8 06:44:25.192: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-2 -- /bin/sh -x -c find /data'
Jan 8 06:44:26.980: INFO: stderr: "+ find /data\n"
Jan 8 06:44:26.980: INFO: stdout: "/data\n/data/statefulset-continue\n/data/lost+found\n"
Jan 8 06:44:26.980: INFO: stdout of find /data on ss-2: /data
/data/statefulset-continue
/data/lost+found
Jan 8 06:44:27.157: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-0 -- /bin/sh -x -c touch /data/1673160255824822651'
Jan 8 06:44:28.934: INFO: stderr: "+ touch /data/1673160255824822651\n"
Jan 8 06:44:28.934: INFO: stdout: ""
Jan 8 06:44:28.934: INFO: stdout of touch /data/1673160255824822651 on ss-0:
Jan 8 06:44:28.934: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-1 -- /bin/sh -x -c touch /data/1673160255824822651'
Jan 8 06:44:30.715: INFO: stderr: "+ touch /data/1673160255824822651\n"
Jan 8 06:44:30.715: INFO: stdout: ""
Jan 8 06:44:30.715: INFO: stdout of touch /data/1673160255824822651 on ss-1:
Jan 8 06:44:30.715: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-2 -- /bin/sh -x -c touch /data/1673160255824822651'
Jan 8 06:44:32.466: INFO: stderr: "+ touch /data/1673160255824822651\n"
Jan 8 06:44:32.466: INFO: stdout: ""
Jan 8 06:44:32.466: INFO: stdout of touch /data/1673160255824822651 on ss-2:
�[1mSTEP:�[0m Verifying statefulset provides a stable hostname for each pod �[38;5;243m01/08/23 06:44:32.466�[0m
Jan 8 06:44:32.643: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-0 -- /bin/sh -x -c printf $(hostname)'
Jan 8 06:44:34.561: INFO: stderr: "+ hostname\n+ printf ss-0\n"
Jan 8 06:44:34.561: INFO: stdout: "ss-0"
Jan 8 06:44:34.561: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-1 -- /bin/sh -x -c printf $(hostname)'
Jan 8 06:44:36.665: INFO: stderr: "+ hostname\n+ printf ss-1\n"
Jan 8 06:44:36.665: INFO: stdout: "ss-1"
Jan 8 06:44:36.665: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-2 -- /bin/sh -x -c printf $(hostname)'
Jan 8 06:44:38.536: INFO: stderr: "+ hostname\n+ printf ss-2\n"
Jan 8 06:44:38.536: INFO: stdout: "ss-2"
�[1mSTEP:�[0m Verifying statefulset set proper service name �[38;5;243m01/08/23 06:44:38.536�[0m
Jan 8 06:44:38.536: INFO: Checking if statefulset spec.serviceName is test
�[1mSTEP:�[0m Running echo $(hostname) | dd of=/data/hostname conv=fsync in all stateful pods �[38;5;243m01/08/23 06:44:38.536�[0m
Jan 8 06:44:38.714: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-0 -- /bin/sh -x -c echo $(hostname) | dd of=/data/hostname conv=fsync'
Jan 8 06:44:40.541: INFO: stderr: "+ dd 'of=/data/hostname' 'conv=fsync'\n+ hostname\n+ echo ss-0\n0+1 records in\n0+1 records out\n"
Jan 8 06:44:40.541: INFO: stdout: ""
Jan 8 06:44:40.541: INFO: stdout of echo $(hostname) | dd of=/data/hostname conv=fsync on ss-0:
Jan 8 06:44:40.541: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-1 -- /bin/sh -x -c echo $(hostname) | dd of=/data/hostname conv=fsync'
Jan 8 06:44:42.383: INFO: stderr: "+ dd 'of=/data/hostname' 'conv=fsync'\n+ hostname\n+ echo ss-1\n0+1 records in\n0+1 records out\n"
Jan 8 06:44:42.383: INFO: stdout: ""
Jan 8 06:44:42.383: INFO: stdout of echo $(hostname) | dd of=/data/hostname conv=fsync on ss-1:
Jan 8 06:44:42.383: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 exec ss-2 -- /bin/sh -x -c echo $(hostname) | dd of=/data/hostname conv=fsync'
Jan 8 06:44:44.170: INFO: stderr: "+ dd 'of=/data/hostname' 'conv=fsync'\n+ hostname\n+ echo ss-2\n0+1 records in\n0+1 records out\n"
Jan 8 06:44:44.170: INFO: stdout: ""
Jan 8 06:44:44.170: INFO: stdout of echo $(hostname) | dd of=/data/hostname conv=fsync on ss-2:
�[1mSTEP:�[0m Restarting statefulset ss �[38;5;243m01/08/23 06:44:44.17�[0m
Jan 8 06:44:44.171: INFO: Scaling statefulset ss to 0
Jan 8 06:44:54.882: INFO: Waiting for statefulset status.replicas updated to 0
Jan 8 06:44:55.596: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:45:05.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:45:15.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:45:25.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:45:35.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:45:45.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:45:55.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:46:05.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:46:15.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:46:25.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:46:35.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:46:45.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:46:55.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:47:05.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:47:15.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:47:25.775: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:47:35.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:47:45.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:47:55.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:48:05.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:48:15.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:48:25.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:48:35.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:48:45.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:48:55.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:49:05.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:49:15.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:49:25.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:49:35.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:49:45.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:49:55.775: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:50:05.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:50:15.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:50:25.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:50:35.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:50:45.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:50:55.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:51:05.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:51:15.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:51:25.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:51:35.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:51:45.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:51:55.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:52:05.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:52:15.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:52:25.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:52:35.775: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:52:45.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:52:55.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:53:05.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:53:15.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:53:25.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:53:35.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:53:45.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:53:55.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:54:05.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:54:15.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:54:25.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:54:35.773: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:54:45.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:54:55.774: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:54:55.951: INFO: Found 1 stateful pods, waiting for 3
Jan 8 06:54:55.951: FAIL: Failed waiting for pods to enter running: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801e128?, 0xc003e7e680}, 0x3, 0x3, 0xc002a03900)
test/e2e/framework/statefulset/wait.go:58 +0xf9
k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
test/e2e/framework/statefulset/wait.go:80
k8s.io/kubernetes/test/e2e/apps.glob..func10.2.3()
test/e2e/apps/statefulset.go:161 +0x41e
[AfterEach] Basic StatefulSet functionality [StatefulSetBasic]
test/e2e/apps/statefulset.go:124
Jan 8 06:54:56.128: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 describe po ss-0'
Jan 8 06:54:57.141: INFO: stderr: ""
Jan 8 06:54:57.141: INFO: stdout: "Name: ss-0\nNamespace: statefulset-4697\nPriority: 0\nService Account: default\nNode: i-0685adc783a822be6/172.20.40.112\nStart Time: Sun, 08 Jan 2023 06:44:55 +0000\nLabels: baz=blah\n controller-revision-hash=ss-b9bbc7d7b\n foo=bar\n statefulset.kubernetes.io/pod-name=ss-0\nAnnotations: <none>\nStatus: Pending\nIP: \nIPs: <none>\nControlled By: StatefulSet/ss\nContainers:\n webserver:\n Container ID: \n Image: registry.k8s.io/e2e-test-images/httpd:2.4.38-4\n Image ID: \n Port: <none>\n Host Port: <none>\n State: Waiting\n Reason: ContainerCreating\n Ready: False\n Restart Count: 0\n Readiness: exec [test -f /data/statefulset-continue] delay=0s timeout=1s period=1s #success=1 #failure=1\n Environment: <none>\n Mounts:\n /data/ from datadir (rw)\n /home from home (rw)\n /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-jmbqd (ro)\nConditions:\n Type Status\n Initialized True \n Ready False \n ContainersReady False \n PodScheduled True \nVolumes:\n datadir:\n Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)\n ClaimName: datadir-ss-0\n ReadOnly: false\n home:\n Type: HostPath (bare host directory volume)\n Path: /tmp/home\n HostPathType: \n kube-api-access-jmbqd:\n Type: Projected (a volume that contains injected data from multiple sources)\n TokenExpirationSeconds: 3607\n ConfigMapName: kube-root-ca.crt\n ConfigMapOptional: <nil>\n DownwardAPI: true\nQoS Class: BestEffort\nNode-Selectors: <none>\nTolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s\n node.kubernetes.io/unreachable:NoExecute op=Exists for 300s\nEvents:\n Type Reason Age From Message\n ---- ------ ---- ---- -------\n Normal Scheduled 10m default-scheduler Successfully assigned statefulset-4697/ss-0 to i-0685adc783a822be6\n Warning FailedAttachVolume 10m attachdetach-controller Multi-Attach error for volume \"pvc-adf59051-191c-45c9-816e-e6310ee8c620\" Volume is already exclusively attached to one node and can't be attached to another\n Normal SuccessfulAttachVolume 9m51s attachdetach-controller AttachVolume.Attach succeeded for volume \"pvc-adf59051-191c-45c9-816e-e6310ee8c620\"\n Warning FailedCreatePodSandBox 9m49s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox \"870642f14d30d3b5085054deab710a24fc40cf662a8adc1e2a7084a35d439139\": plugin type=\"cilium-cni\" name=\"cilium\" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available\n Warning FailedCreatePodSandBox 9m37s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox \"53d0bdfb58baa7767d88e7d02fe46d106eca150881076794eaa56967e755d46a\": plugin type=\"cilium-cni\" name=\"cilium\" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available\n Warning FailedCreatePodSandBox 9m25s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox \"f23286731bf78576e621b6721b472b154867251ab8834bbbcb7bc651498d9a97\": plugin type=\"cilium-cni\" name=\"cilium\" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available\n Warning FailedCreatePodSandBox 9m12s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox \"9edc303ec86d31a74a677876fbeb196b575af68d12daa65fceea494dd323c60f\": plugin type=\"cilium-cni\" name=\"cilium\" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available\n Warning FailedCreatePodSandBox 8m57s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox \"3bbd577f0eba6f6e12e99b310d3caaf04e498ea66d15dd37f938d2aa6d1524fc\": plugin type=\"cilium-cni\" name=\"cilium\" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available\n Warning FailedCreatePodSandBox 8m43s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox \"dca811c4b8765a37766cd31b2a80a8f03c6a947d3ed4b05ddb4d103050a702f0\": plugin type=\"cilium-cni\" name=\"cilium\" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available\n Warning FailedCreatePodSandBox 8m28s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox \"e41bda3abe6ac71127d587ef82c12f908dde7ad4346ba7052785eaaa2d65c0a8\": plugin type=\"cilium-cni\" name=\"cilium\" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available\n Warning FailedCreatePodSandBox 8m12s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox \"ec65c2a17229409a66d33ac2daefd897ff910cce3b2557c19c80290205a0c575\": plugin type=\"cilium-cni\" name=\"cilium\" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available\n Warning FailedCreatePodSandBox 7m57s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox \"49fceb4a804fc8709f6554133b017e823edd523b94318b4c7680c74658aa6839\": plugin type=\"cilium-cni\" name=\"cilium\" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available\n Warning FailedCreatePodSandBox 4m16s (x17 over 7m45s) kubelet (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox \"cb7fd013bccd49a1279c602a5d64d01f26d1d9ccba513b4b9b12553b5c138e2b\": plugin type=\"cilium-cni\" name=\"cilium\" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available\n"
Jan 8 06:54:57.141: INFO:
Output of kubectl describe ss-0:
Name: ss-0
Namespace: statefulset-4697
Priority: 0
Service Account: default
Node: i-0685adc783a822be6/172.20.40.112
Start Time: Sun, 08 Jan 2023 06:44:55 +0000
Labels: baz=blah
controller-revision-hash=ss-b9bbc7d7b
foo=bar
statefulset.kubernetes.io/pod-name=ss-0
Annotations: <none>
Status: Pending
IP:
IPs: <none>
Controlled By: StatefulSet/ss
Containers:
webserver:
Container ID:
Image: registry.k8s.io/e2e-test-images/httpd:2.4.38-4
Image ID:
Port: <none>
Host Port: <none>
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Readiness: exec [test -f /data/statefulset-continue] delay=0s timeout=1s period=1s #success=1 #failure=1
Environment: <none>
Mounts:
/data/ from datadir (rw)
/home from home (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-jmbqd (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
datadir:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: datadir-ss-0
ReadOnly: false
home:
Type: HostPath (bare host directory volume)
Path: /tmp/home
HostPathType:
kube-api-access-jmbqd:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 10m default-scheduler Successfully assigned statefulset-4697/ss-0 to i-0685adc783a822be6
Warning FailedAttachVolume 10m attachdetach-controller Multi-Attach error for volume "pvc-adf59051-191c-45c9-816e-e6310ee8c620" Volume is already exclusively attached to one node and can't be attached to another
Normal SuccessfulAttachVolume 9m51s attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-adf59051-191c-45c9-816e-e6310ee8c620"
Warning FailedCreatePodSandBox 9m49s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "870642f14d30d3b5085054deab710a24fc40cf662a8adc1e2a7084a35d439139": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Warning FailedCreatePodSandBox 9m37s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "53d0bdfb58baa7767d88e7d02fe46d106eca150881076794eaa56967e755d46a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Warning FailedCreatePodSandBox 9m25s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f23286731bf78576e621b6721b472b154867251ab8834bbbcb7bc651498d9a97": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Warning FailedCreatePodSandBox 9m12s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9edc303ec86d31a74a677876fbeb196b575af68d12daa65fceea494dd323c60f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Warning FailedCreatePodSandBox 8m57s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3bbd577f0eba6f6e12e99b310d3caaf04e498ea66d15dd37f938d2aa6d1524fc": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Warning FailedCreatePodSandBox 8m43s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "dca811c4b8765a37766cd31b2a80a8f03c6a947d3ed4b05ddb4d103050a702f0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Warning FailedCreatePodSandBox 8m28s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e41bda3abe6ac71127d587ef82c12f908dde7ad4346ba7052785eaaa2d65c0a8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Warning FailedCreatePodSandBox 8m12s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ec65c2a17229409a66d33ac2daefd897ff910cce3b2557c19c80290205a0c575": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Warning FailedCreatePodSandBox 7m57s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "49fceb4a804fc8709f6554133b017e823edd523b94318b4c7680c74658aa6839": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Warning FailedCreatePodSandBox 4m16s (x17 over 7m45s) kubelet (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cb7fd013bccd49a1279c602a5d64d01f26d1d9ccba513b4b9b12553b5c138e2b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:54:57.141: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=statefulset-4697 logs ss-0 --tail=100'
Jan 8 06:54:57.951: INFO: rc: 1
Jan 8 06:54:57.951: INFO:
Last 100 log lines of ss-0:
Jan 8 06:54:57.951: INFO: Deleting all statefulset in ns statefulset-4697
Jan 8 06:54:58.128: INFO: Scaling statefulset ss to 0
Jan 8 06:55:08.838: INFO: Waiting for statefulset status.replicas updated to 0
Jan 8 06:55:09.014: INFO: Deleting statefulset ss
Jan 8 06:55:09.371: INFO: Deleting pvc: datadir-ss-0 with volume pvc-adf59051-191c-45c9-816e-e6310ee8c620
Jan 8 06:55:09.550: INFO: Deleting pvc: datadir-ss-1 with volume pvc-fb1389ed-b9df-495b-aba1-c8a0b85caacd
Jan 8 06:55:09.728: INFO: Deleting pvc: datadir-ss-2 with volume pvc-753e64b2-4132-4a3d-8ba6-5e4ef8f7d989
Jan 8 06:55:10.084: INFO: Still waiting for pvs of statefulset to disappear:
pvc-adf59051-191c-45c9-816e-e6310ee8c620: {Phase:Released Message: Reason:}
[AfterEach] [sig-apps] StatefulSet
test/e2e/framework/node/init/init.go:32
Jan 8 06:55:20.261: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-apps] StatefulSet
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-apps] StatefulSet
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/08/23 06:55:20.439�[0m
�[1mSTEP:�[0m Collecting events from namespace "statefulset-4697". �[38;5;243m01/08/23 06:55:20.439�[0m
�[1mSTEP:�[0m Found 55 events. �[38;5;243m01/08/23 06:55:20.617�[0m
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:42:48 +0000 UTC - event for datadir-ss-0: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:42:48 +0000 UTC - event for datadir-ss-0: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:42:48 +0000 UTC - event for datadir-ss-0: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-vzzj9_eb492b31-b783-48e6-ab2b-bc22df496b4f } Provisioning: External provisioner is provisioning volume for claim "statefulset-4697/datadir-ss-0"
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:42:48 +0000 UTC - event for ss: {statefulset-controller } SuccessfulCreate: create Pod ss-0 in StatefulSet ss successful
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:42:48 +0000 UTC - event for ss: {statefulset-controller } SuccessfulCreate: create Claim datadir-ss-0 Pod ss-0 in StatefulSet ss success
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:42:52 +0000 UTC - event for datadir-ss-0: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-vzzj9_eb492b31-b783-48e6-ab2b-bc22df496b4f } ProvisioningSucceeded: Successfully provisioned volume pvc-adf59051-191c-45c9-816e-e6310ee8c620
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:42:52 +0000 UTC - event for ss-0: {default-scheduler } Scheduled: Successfully assigned statefulset-4697/ss-0 to i-0bd196cab651da520
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:42:55 +0000 UTC - event for ss-0: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-adf59051-191c-45c9-816e-e6310ee8c620"
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:05 +0000 UTC - event for ss-0: {kubelet i-0bd196cab651da520} Created: Created container webserver
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:05 +0000 UTC - event for ss-0: {kubelet i-0bd196cab651da520} Started: Started container webserver
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:05 +0000 UTC - event for ss-0: {kubelet i-0bd196cab651da520} Pulled: Container image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4" already present on machine
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:06 +0000 UTC - event for ss-0: {kubelet i-0bd196cab651da520} Unhealthy: Readiness probe failed:
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:14 +0000 UTC - event for datadir-ss-1: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-vzzj9_eb492b31-b783-48e6-ab2b-bc22df496b4f } Provisioning: External provisioner is provisioning volume for claim "statefulset-4697/datadir-ss-1"
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:14 +0000 UTC - event for datadir-ss-1: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:14 +0000 UTC - event for datadir-ss-1: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:14 +0000 UTC - event for ss: {statefulset-controller } SuccessfulCreate: create Claim datadir-ss-1 Pod ss-1 in StatefulSet ss success
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:14 +0000 UTC - event for ss: {statefulset-controller } SuccessfulCreate: create Pod ss-1 in StatefulSet ss successful
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:18 +0000 UTC - event for datadir-ss-1: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-vzzj9_eb492b31-b783-48e6-ab2b-bc22df496b4f } ProvisioningSucceeded: Successfully provisioned volume pvc-fb1389ed-b9df-495b-aba1-c8a0b85caacd
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:18 +0000 UTC - event for ss-1: {default-scheduler } Scheduled: Successfully assigned statefulset-4697/ss-1 to i-0d149644e2edcf9cf
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:26 +0000 UTC - event for ss-1: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-fb1389ed-b9df-495b-aba1-c8a0b85caacd"
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:31 +0000 UTC - event for ss-1: {kubelet i-0d149644e2edcf9cf} Started: Started container webserver
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:31 +0000 UTC - event for ss-1: {kubelet i-0d149644e2edcf9cf} Pulled: Container image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4" already present on machine
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:31 +0000 UTC - event for ss-1: {kubelet i-0d149644e2edcf9cf} Created: Created container webserver
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:31 +0000 UTC - event for ss-1: {kubelet i-0d149644e2edcf9cf} Unhealthy: Readiness probe failed:
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:50 +0000 UTC - event for datadir-ss-2: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-vzzj9_eb492b31-b783-48e6-ab2b-bc22df496b4f } Provisioning: External provisioner is provisioning volume for claim "statefulset-4697/datadir-ss-2"
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:50 +0000 UTC - event for datadir-ss-2: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:50 +0000 UTC - event for datadir-ss-2: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:50 +0000 UTC - event for ss: {statefulset-controller } SuccessfulCreate: create Claim datadir-ss-2 Pod ss-2 in StatefulSet ss success
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:50 +0000 UTC - event for ss: {statefulset-controller } SuccessfulCreate: create Pod ss-2 in StatefulSet ss successful
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:54 +0000 UTC - event for datadir-ss-2: {ebs.csi.aws.com_ebs-csi-controller-89b649c94-vzzj9_eb492b31-b783-48e6-ab2b-bc22df496b4f } ProvisioningSucceeded: Successfully provisioned volume pvc-753e64b2-4132-4a3d-8ba6-5e4ef8f7d989
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:54 +0000 UTC - event for ss-2: {default-scheduler } Scheduled: Successfully assigned statefulset-4697/ss-2 to i-0685adc783a822be6
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:43:57 +0000 UTC - event for ss-2: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-753e64b2-4132-4a3d-8ba6-5e4ef8f7d989"
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:44:03 +0000 UTC - event for ss-2: {kubelet i-0685adc783a822be6} Pulled: Container image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4" already present on machine
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:44:03 +0000 UTC - event for ss-2: {kubelet i-0685adc783a822be6} Started: Started container webserver
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:44:03 +0000 UTC - event for ss-2: {kubelet i-0685adc783a822be6} Created: Created container webserver
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:44:04 +0000 UTC - event for ss-2: {kubelet i-0685adc783a822be6} Unhealthy: Readiness probe failed:
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:44:44 +0000 UTC - event for ss: {statefulset-controller } SuccessfulDelete: delete Pod ss-2 in StatefulSet ss successful
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:44:44 +0000 UTC - event for ss-2: {kubelet i-0685adc783a822be6} Killing: Stopping container webserver
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:44:48 +0000 UTC - event for ss: {statefulset-controller } SuccessfulDelete: delete Pod ss-1 in StatefulSet ss successful
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:44:48 +0000 UTC - event for ss-1: {kubelet i-0d149644e2edcf9cf} Killing: Stopping container webserver
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:44:50 +0000 UTC - event for ss: {statefulset-controller } SuccessfulDelete: delete Pod ss-0 in StatefulSet ss successful
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:44:50 +0000 UTC - event for ss-0: {kubelet i-0bd196cab651da520} Killing: Stopping container webserver
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:44:55 +0000 UTC - event for ss-0: {default-scheduler } Scheduled: Successfully assigned statefulset-4697/ss-0 to i-0685adc783a822be6
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:44:55 +0000 UTC - event for ss-0: {attachdetach-controller } FailedAttachVolume: Multi-Attach error for volume "pvc-adf59051-191c-45c9-816e-e6310ee8c620" Volume is already exclusively attached to one node and can't be attached to another
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:45:06 +0000 UTC - event for ss-0: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-adf59051-191c-45c9-816e-e6310ee8c620"
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:45:08 +0000 UTC - event for ss-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "870642f14d30d3b5085054deab710a24fc40cf662a8adc1e2a7084a35d439139": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:45:20 +0000 UTC - event for ss-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "53d0bdfb58baa7767d88e7d02fe46d106eca150881076794eaa56967e755d46a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:45:32 +0000 UTC - event for ss-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f23286731bf78576e621b6721b472b154867251ab8834bbbcb7bc651498d9a97": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:45:45 +0000 UTC - event for ss-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9edc303ec86d31a74a677876fbeb196b575af68d12daa65fceea494dd323c60f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:46:00 +0000 UTC - event for ss-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3bbd577f0eba6f6e12e99b310d3caaf04e498ea66d15dd37f938d2aa6d1524fc": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:46:14 +0000 UTC - event for ss-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "dca811c4b8765a37766cd31b2a80a8f03c6a947d3ed4b05ddb4d103050a702f0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:46:29 +0000 UTC - event for ss-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e41bda3abe6ac71127d587ef82c12f908dde7ad4346ba7052785eaaa2d65c0a8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:46:45 +0000 UTC - event for ss-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ec65c2a17229409a66d33ac2daefd897ff910cce3b2557c19c80290205a0c575": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:47:00 +0000 UTC - event for ss-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "49fceb4a804fc8709f6554133b017e823edd523b94318b4c7680c74658aa6839": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:55:20.617: INFO: At 2023-01-08 06:47:12 +0000 UTC - event for ss-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cb7fd013bccd49a1279c602a5d64d01f26d1d9ccba513b4b9b12553b5c138e2b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:55:20.794: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 8 06:55:20.794: INFO:
Jan 8 06:55:20.972: INFO:
Logging node info for node i-0685adc783a822be6
Jan 8 06:55:21.148: INFO: Node Info: &Node{ObjectMeta:{i-0685adc783a822be6 3eb588c7-cc76-44b4-b28a-492b9989254b 43119 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0685adc783a822be6 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0685adc783a822be6 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.112 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0685adc783a822be6"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:55:04 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-08 06:55:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0685adc783a822be6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:04 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:04 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:04 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:55:04 +0000 UTC,LastTransitionTime:2023-01-08 06:25:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.112,},NodeAddress{Type:ExternalIP,Address:43.201.46.149,},NodeAddress{Type:InternalDNS,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-201-46-149.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dccafbf198926b990228825ae0608,SystemUUID:ec2dccaf-bf19-8926-b990-228825ae0608,BootID:b8ff627c-af94-4730-be1d-6c7bfbfc818f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-025c7f75403571919 kubernetes.io/csi/ebs.csi.aws.com^vol-0488b0c43122e11c1 kubernetes.io/csi/ebs.csi.aws.com^vol-0b8657a4eb6501329],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-025c7f75403571919,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0b8657a4eb6501329,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0488b0c43122e11c1,DevicePath:,},},Config:nil,},}
Jan 8 06:55:21.149: INFO:
Logging kubelet events for node i-0685adc783a822be6
Jan 8 06:55:21.328: INFO:
Logging pods the kubelet thinks is on node i-0685adc783a822be6
Jan 8 06:55:21.519: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-zgmck started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:21.519: INFO: cilium-9vhbx started at 2023-01-08 06:25:38 +0000 UTC (1+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 8 06:55:21.519: INFO: Container cilium-agent ready: true, restart count 0
Jan 8 06:55:21.519: INFO: pod-disruption-failure-ignore-1-kfqqr started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container c ready: false, restart count 0
Jan 8 06:55:21.519: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-gq5pr started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:21.519: INFO: service-headless-bsxnn started at 2023-01-08 06:50:25 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container service-headless ready: false, restart count 0
Jan 8 06:55:21.519: INFO: inline-volume-tester-9qdkl started at 2023-01-08 06:52:11 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 8 06:55:21.519: INFO: csi-mockplugin-0 started at 2023-01-08 06:50:39 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container csi-provisioner ready: false, restart count 0
Jan 8 06:55:21.519: INFO: Container driver-registrar ready: false, restart count 0
Jan 8 06:55:21.519: INFO: Container mock ready: false, restart count 0
Jan 8 06:55:21.519: INFO: pod-subpath-test-inlinevolume-plpm started at 2023-01-08 06:52:37 +0000 UTC (2+2 container statuses recorded)
Jan 8 06:55:21.519: INFO: Init container init-volume-inlinevolume-plpm ready: false, restart count 0
Jan 8 06:55:21.519: INFO: Init container test-init-subpath-inlinevolume-plpm ready: false, restart count 0
Jan 8 06:55:21.519: INFO: Container test-container-subpath-inlinevolume-plpm ready: false, restart count 0
Jan 8 06:55:21.519: INFO: Container test-container-volume-inlinevolume-plpm ready: false, restart count 0
Jan 8 06:55:21.519: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:51:13 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container csi-attacher ready: false, restart count 0
Jan 8 06:55:21.519: INFO: pod-937b6812-ddb0-47fe-af8b-07288c742e73 started at 2023-01-08 06:52:10 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container write-pod ready: false, restart count 0
Jan 8 06:55:21.519: INFO: inline-volume-tester-g992k started at 2023-01-08 06:44:18 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container csi-volume-tester ready: true, restart count 0
Jan 8 06:55:21.519: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-svbp2 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:21.519: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mpw58 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:21.519: INFO: service-proxy-toggled-dh67n started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container service-proxy-toggled ready: false, restart count 0
Jan 8 06:55:21.519: INFO: liveness-51325787-aaa2-4ac1-a3f9-8ff47e3fa793 started at 2023-01-08 06:54:56 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container agnhost-container ready: false, restart count 0
Jan 8 06:55:21.519: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rrmzm started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:21.519: INFO: busybox1 started at 2023-01-08 06:51:29 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container busybox ready: false, restart count 0
Jan 8 06:55:21.519: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:50:39 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container csi-attacher ready: false, restart count 0
Jan 8 06:55:21.519: INFO: pod-disruption-failure-ignore-0-w4rvn started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container c ready: false, restart count 0
Jan 8 06:55:21.519: INFO: pod-projected-secrets-907df958-d79f-42f9-87ee-41301a6f40be started at 2023-01-08 06:55:13 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container secret-volume-test ready: false, restart count 0
Jan 8 06:55:21.519: INFO: ebs-csi-node-ft7bw started at 2023-01-08 06:25:38 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:55:21.519: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:55:21.519: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 8 06:55:21.519: INFO: suspend-false-to-true-97wd6 started at 2023-01-08 06:53:18 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container c ready: false, restart count 0
Jan 8 06:55:21.519: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-q97zd started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:21.519: INFO: service-proxy-disabled-6pp8x started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container service-proxy-disabled ready: true, restart count 0
Jan 8 06:55:21.519: INFO: pod-8fb38c56-a909-49a3-85f0-9d73b5275577 started at 2023-01-08 06:50:50 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container test-container ready: false, restart count 0
Jan 8 06:55:21.519: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2bn4h started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:21.519: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v9tjx started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:21.519: INFO: affinity-clusterip-zxklh started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container affinity-clusterip ready: false, restart count 0
Jan 8 06:55:21.519: INFO: csi-mockplugin-0 started at 2023-01-08 06:51:13 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container csi-provisioner ready: false, restart count 0
Jan 8 06:55:21.519: INFO: Container driver-registrar ready: false, restart count 0
Jan 8 06:55:21.519: INFO: Container mock ready: false, restart count 0
Jan 8 06:55:21.519: INFO: agnhost-primary-9vrbd started at 2023-01-08 06:54:54 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container agnhost-primary ready: false, restart count 0
Jan 8 06:55:21.519: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-97qp6 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:21.519: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-4d46c started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:21.519: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:22.154: INFO:
Latency metrics for node i-0685adc783a822be6
Jan 8 06:55:22.154: INFO:
Logging node info for node i-0bd196cab651da520
Jan 8 06:55:22.331: INFO: Node Info: &Node{ObjectMeta:{i-0bd196cab651da520 f7f00780-d24e-4c8e-b54d-373598e1d1d2 42453 0 2023-01-08 06:26:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0bd196cab651da520 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0bd196cab651da520 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.104 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bd196cab651da520"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:26:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:52:47 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-08 06:52:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0bd196cab651da520,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:52:46 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:52:46 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:52:46 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:52:46 +0000 UTC,LastTransitionTime:2023-01-08 06:26:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.104,},NodeAddress{Type:ExternalIP,Address:54.180.1.55,},NodeAddress{Type:InternalDNS,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-180-1-55.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec218f9d32a84b2feeb6301255fd576c,SystemUUID:ec218f9d-32a8-4b2f-eeb6-301255fd576c,BootID:accdf7fd-1195-4765-b28e-29d44bf47bac,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-06c6e766891904b0f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-06c6e766891904b0f,DevicePath:,},},Config:nil,},}
Jan 8 06:55:22.331: INFO:
Logging kubelet events for node i-0bd196cab651da520
Jan 8 06:55:22.510: INFO:
Logging pods the kubelet thinks is on node i-0bd196cab651da520
Jan 8 06:55:22.694: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-76nxw started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:22.694: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-75r8w started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:22.694: INFO: pod-a15c4150-c88e-49b1-a111-7f2271cf819c started at 2023-01-08 06:46:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container write-pod ready: true, restart count 0
Jan 8 06:55:22.694: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v44bk started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:22.694: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9mp69 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:22.694: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-cddx5 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:22.694: INFO: suspend-false-to-true-xwpqd started at 2023-01-08 06:53:18 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container c ready: false, restart count 0
Jan 8 06:55:22.694: INFO: pod-projected-configmaps-143d6d2e-7b9b-47e8-97c7-c2b8adc28191 started at 2023-01-08 06:52:29 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container agnhost-container ready: false, restart count 0
Jan 8 06:55:22.694: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-6df8l started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:22.694: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-vrcjq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:22.694: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9776c started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:22.694: INFO: ebs-csi-node-6grlm started at 2023-01-08 06:26:02 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:55:22.694: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:55:22.694: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 8 06:55:22.694: INFO: webserver-pod started at 2023-01-08 06:46:47 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container agnhost ready: false, restart count 0
Jan 8 06:55:22.694: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9wskq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:22.694: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mphgx started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:22.694: INFO: pod1 started at 2023-01-08 06:51:10 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container agnhost-container ready: false, restart count 0
Jan 8 06:55:22.694: INFO: affinity-clusterip-mrptn started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container affinity-clusterip ready: false, restart count 0
Jan 8 06:55:22.694: INFO: cilium-s6shx started at 2023-01-08 06:26:02 +0000 UTC (1+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 8 06:55:22.694: INFO: Container cilium-agent ready: true, restart count 0
Jan 8 06:55:22.694: INFO: service-proxy-toggled-8nfkz started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container service-proxy-toggled ready: false, restart count 0
Jan 8 06:55:22.694: INFO: service-headless-x4nfw started at 2023-01-08 06:50:25 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:22.694: INFO: Container service-headless ready: false, restart count 0
Jan 8 06:55:23.326: INFO:
Latency metrics for node i-0bd196cab651da520
Jan 8 06:55:23.326: INFO:
Logging node info for node i-0d149644e2edcf9cf
Jan 8 06:55:23.504: INFO: Node Info: &Node{ObjectMeta:{i-0d149644e2edcf9cf b448ef72-6e61-4ad5-aa29-d63cc0f94945 41768 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0d149644e2edcf9cf kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0d149644e2edcf9cf topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.115 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0d149644e2edcf9cf"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:51:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0d149644e2edcf9cf,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:51:18 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:51:18 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:51:18 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:51:18 +0000 UTC,LastTransitionTime:2023-01-08 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.115,},NodeAddress{Type:ExternalIP,Address:3.35.219.218,},NodeAddress{Type:InternalDNS,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-35-219-218.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec26c6a9af66729e278545162595eb2d,SystemUUID:ec26c6a9-af66-729e-2785-45162595eb2d,BootID:959da9e3-fb86-442a-958f-2a49c509cdf4,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 8 06:55:23.504: INFO:
Logging kubelet events for node i-0d149644e2edcf9cf
Jan 8 06:55:23.684: INFO:
Logging pods the kubelet thinks is on node i-0d149644e2edcf9cf
Jan 8 06:55:23.868: INFO: probe-test-8a476652-ee98-415a-b6f6-009d080c5f05 started at 2023-01-08 06:51:44 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:23.868: INFO: Container probe-test-8a476652-ee98-415a-b6f6-009d080c5f05 ready: false, restart count 0
Jan 8 06:55:23.868: INFO: csi-mockplugin-0 started at 2023-01-08 06:52:13 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:55:23.868: INFO: Container csi-provisioner ready: false, restart count 0
Jan 8 06:55:23.868: INFO: Container driver-registrar ready: false, restart count 0
Jan 8 06:55:23.868: INFO: Container mock ready: false, restart count 0
Jan 8 06:55:23.868: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-njh2f started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:23.868: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:23.868: INFO: cilium-58twl started at 2023-01-08 06:25:37 +0000 UTC (1+1 container statuses recorded)
Jan 8 06:55:23.868: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 8 06:55:23.868: INFO: Container cilium-agent ready: true, restart count 0
Jan 8 06:55:23.868: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rv5qq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:23.868: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:23.868: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-flhps started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:23.868: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:23.869: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2vh6v started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:23.869: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:23.869: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nt9jr started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:23.869: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0
Jan 8 06:55:23.869: INFO: ebs-csi-node-lfcwb started at 2023-01-08 06:25:37 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:55:23.869: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:55:23.869: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:55:23.869: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 8 06:55:23.869: INFO: coredns-559769c974-9q4qg started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:23.869: INFO: Container coredns ready: true, restart count 0
Jan 8 06:55:23.869: INFO: csi-hostpathplugin-0 started at 2023-01-08 06:52:05 +0000 UTC (0+7 container statuses recorded)
Jan 8 06:55:23.869: INFO: Container csi-attacher ready: false, restart count 0
Jan 8 06:55:23.869: INFO: Container csi-provisioner ready: false, restart count 0
Jan 8 06:55:23.869: INFO: Container csi-resizer ready: false, restart count 0
Jan 8 06:55:23.869: INFO: Container csi-snapshotter ready: false, restart count 0
Jan 8 06:55:23.869: INFO: Container hostpath ready: false, restart count 0
Jan 8 06:55:23.869: INFO: Container liveness-probe ready: false, restart count 0
Jan 8 06:55:23.869: INFO: Container node-driver-registrar ready: false, restart count 0
Jan 8 06:55:23.869: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-xrmq4 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:23.869: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:23.869: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-7gwsq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:23.869: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:23.869: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nsfnm started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:23.869: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:23.869: INFO: coredns-autoscaler-7cb5c5b969-xqnk5 started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:23.869: INFO: Container autoscaler ready: true, restart count 0
Jan 8 06:55:23.869: INFO: service-proxy-disabled-v5f82 started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:23.869: INFO: Container service-proxy-disabled ready: true, restart count 0
Jan 8 06:55:23.869: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-tgn6r started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:23.869: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:23.869: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:52:13 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:23.869: INFO: Container csi-attacher ready: false, restart count 0
Jan 8 06:55:23.869: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9b2xq started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:23.869: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:24.482: INFO:
Latency metrics for node i-0d149644e2edcf9cf
Jan 8 06:55:24.482: INFO:
Logging node info for node i-0eee6113fd9f81d94
Jan 8 06:55:24.659: INFO: Node Info: &Node{ObjectMeta:{i-0eee6113fd9f81d94 6de592fc-f11c-4ba3-afa0-707bb4ca29b2 41943 0 2023-01-08 06:23:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0eee6113fd9f81d94 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.48.64 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eee6113fd9f81d94"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-08 06:23:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-08 06:24:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-08 06:24:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-08 06:51:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0eee6113fd9f81d94,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:51:55 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:51:55 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:51:55 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:51:55 +0000 UTC,LastTransitionTime:2023-01-08 06:24:35 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.48.64,},NodeAddress{Type:ExternalIP,Address:3.35.206.95,},NodeAddress{Type:InternalDNS,Address:i-0eee6113fd9f81d94.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0eee6113fd9f81d94.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-35-206-95.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d8cf21861b15bb593210ff59f6044,SystemUUID:ec2d8cf2-1861-b15b-b593-210ff59f6044,BootID:fcad947c-a63f-4722-9a63-22eac74bea9f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43453935,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42803460,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967354,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 8 06:55:24.659: INFO:
Logging kubelet events for node i-0eee6113fd9f81d94
Jan 8 06:55:24.838: INFO:
Logging pods the kubelet thinks is on node i-0eee6113fd9f81d94
Jan 8 06:55:25.021: INFO: cilium-f22px started at 2023-01-08 06:24:21 +0000 UTC (1+1 container statuses recorded)
Jan 8 06:55:25.021: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 8 06:55:25.021: INFO: Container cilium-agent ready: true, restart count 0
Jan 8 06:55:25.021: INFO: etcd-manager-events-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:25.021: INFO: Container etcd-manager ready: true, restart count 0
Jan 8 06:55:25.021: INFO: etcd-manager-main-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:25.021: INFO: Container etcd-manager ready: true, restart count 0
Jan 8 06:55:25.021: INFO: kops-controller-242qq started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:25.021: INFO: Container kops-controller ready: true, restart count 0
Jan 8 06:55:25.021: INFO: ebs-csi-node-nspqr started at 2023-01-08 06:24:21 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:55:25.021: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:55:25.021: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:55:25.021: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 8 06:55:25.021: INFO: aws-cloud-controller-manager-d8d56 started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:25.021: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 8 06:55:25.021: INFO: cilium-operator-5dd44dc49f-fd8vl started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:25.021: INFO: Container cilium-operator ready: true, restart count 0
Jan 8 06:55:25.021: INFO: ebs-csi-controller-89b649c94-vzzj9 started at 2023-01-08 06:24:21 +0000 UTC (0+5 container statuses recorded)
Jan 8 06:55:25.021: INFO: Container csi-attacher ready: true, restart count 0
Jan 8 06:55:25.021: INFO: Container csi-provisioner ready: true, restart count 0
Jan 8 06:55:25.021: INFO: Container csi-resizer ready: true, restart count 0
Jan 8 06:55:25.021: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:55:25.021: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:55:25.021: INFO: dns-controller-69987775c6-v4cmg started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:25.021: INFO: Container dns-controller ready: true, restart count 0
Jan 8 06:55:25.021: INFO: kube-scheduler-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:25.021: INFO: Container kube-scheduler ready: true, restart count 0
Jan 8 06:55:25.021: INFO: kube-apiserver-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+2 container statuses recorded)
Jan 8 06:55:25.021: INFO: Container healthcheck ready: true, restart count 0
Jan 8 06:55:25.021: INFO: Container kube-apiserver ready: true, restart count 1
Jan 8 06:55:25.021: INFO: kube-controller-manager-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:25.021: INFO: Container kube-controller-manager ready: true, restart count 2
Jan 8 06:55:25.608: INFO:
Latency metrics for node i-0eee6113fd9f81d94
Jan 8 06:55:25.608: INFO:
Logging node info for node i-0f469f08be563dad4
Jan 8 06:55:25.785: INFO: Node Info: &Node{ObjectMeta:{i-0f469f08be563dad4 fddb222c-a6ac-415f-83dc-4b71acb0e5ab 43138 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0f469f08be563dad4 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0f469f08be563dad4 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.57.173 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0f469f08be563dad4"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:55:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0f469f08be563dad4,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.57.173,},NodeAddress{Type:ExternalIP,Address:3.38.152.241,},NodeAddress{Type:InternalDNS,Address:i-0f469f08be563dad4.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f469f08be563dad4.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-38-152-241.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2abc7b1d816532cfb4787d85cb98ea,SystemUUID:ec2abc7b-1d81-6532-cfb4-787d85cb98ea,BootID:442daa6f-46d1-4c32-9182-8b6204c22aba,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 8 06:55:25.785: INFO:
Logging kubelet events for node i-0f469f08be563dad4
Jan 8 06:55:25.964: INFO:
Logging pods the kubelet thinks is on node i-0f469f08be563dad4
Jan 8 06:55:26.149: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-ktkbb started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:26.149: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-dvpgx started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:26.149: INFO: cilium-24rbm started at 2023-01-08 06:25:37 +0000 UTC (1+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 8 06:55:26.149: INFO: Container cilium-agent ready: true, restart count 0
Jan 8 06:55:26.149: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-x88cb started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:26.149: INFO: ebs-csi-node-9s8qc started at 2023-01-08 06:25:37 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:55:26.149: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:55:26.149: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 8 06:55:26.149: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-h5c5c started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:26.149: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-jc58p started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:26.149: INFO: service-headless-bn7ft started at 2023-01-08 06:50:25 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container service-headless ready: true, restart count 0
Jan 8 06:55:26.149: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-gdlbp started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:26.149: INFO: service-proxy-disabled-bqbxv started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container service-proxy-disabled ready: true, restart count 0
Jan 8 06:55:26.149: INFO: service-proxy-toggled-7qcjj started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container service-proxy-toggled ready: false, restart count 0
Jan 8 06:55:26.149: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-glvzd started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:26.149: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-cwg4t started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:26.149: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-57rwf started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0
Jan 8 06:55:26.149: INFO: pod-configmaps-393a999b-3a18-4df6-95b4-26106a4b0e61 started at 2023-01-08 06:51:56 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container agnhost-container ready: false, restart count 0
Jan 8 06:55:26.149: INFO: coredns-559769c974-f9s4x started at 2023-01-08 06:26:19 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container coredns ready: true, restart count 0
Jan 8 06:55:26.149: INFO: affinity-clusterip-8kclb started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container affinity-clusterip ready: false, restart count 0
Jan 8 06:55:26.149: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-b6m8x started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:55:26.149: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:55:26.871: INFO:
Latency metrics for node i-0f469f08be563dad4
[DeferCleanup (Each)] [sig-apps] StatefulSet
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "statefulset-4697" for this suite. �[38;5;243m01/08/23 06:55:26.871�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sclient\sKubectl\scopy\sshould\scopy\sa\sfile\sfrom\sa\srunning\sPod$'
test/e2e/kubectl/kubectl.go:1537 k8s.io/kubernetes/test/e2e/kubectl.glob..func1.22.1() test/e2e/kubectl/kubectl.go:1537 +0x205from junit_01.xml
[BeforeEach] [sig-cli] Kubectl client set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/08/23 06:51:26.471�[0m Jan 8 06:51:26.471: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename kubectl �[38;5;243m01/08/23 06:51:26.472�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/08/23 06:51:27.005�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/08/23 06:51:27.356�[0m [BeforeEach] [sig-cli] Kubectl client test/e2e/framework/metrics/init/init.go:31 [BeforeEach] [sig-cli] Kubectl client test/e2e/kubectl/kubectl.go:274 [BeforeEach] Kubectl copy test/e2e/kubectl/kubectl.go:1533 �[1mSTEP:�[0m creating the pod �[38;5;243m01/08/23 06:51:27.706�[0m Jan 8 06:51:27.707: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-6574 create -f -' Jan 8 06:51:29.970: INFO: stderr: "" Jan 8 06:51:29.970: INFO: stdout: "pod/busybox1 created\n" Jan 8 06:51:29.970: INFO: Waiting up to 5m0s for 1 pods to be running and ready: [busybox1] Jan 8 06:51:29.970: INFO: Waiting up to 5m0s for pod "busybox1" in namespace "kubectl-6574" to be "running and ready" Jan 8 06:51:30.146: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 176.131831ms Jan 8 06:51:30.146: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:51:32.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2.352429076s Jan 8 06:51:32.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:51:34.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4.353240204s Jan 8 06:51:34.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:51:36.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 6.35287128s Jan 8 06:51:36.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:51:38.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 8.353503752s Jan 8 06:51:38.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:51:40.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 10.353464532s Jan 8 06:51:40.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:51:42.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 12.352771339s Jan 8 06:51:42.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:51:44.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 14.353417605s Jan 8 06:51:44.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:51:46.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 16.352894325s Jan 8 06:51:46.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:51:48.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 18.353781849s Jan 8 06:51:48.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:51:50.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 20.352540315s Jan 8 06:51:50.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:51:52.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 22.352909628s Jan 8 06:51:52.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:51:54.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 24.353512789s Jan 8 06:51:54.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:51:56.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 26.353602114s Jan 8 06:51:56.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:51:58.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 28.353278968s Jan 8 06:51:58.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:00.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 30.353169188s Jan 8 06:52:00.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:02.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 32.352956459s Jan 8 06:52:02.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:04.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 34.353139554s Jan 8 06:52:04.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:06.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 36.35300251s Jan 8 06:52:06.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:08.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 38.353231139s Jan 8 06:52:08.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:10.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 40.352264123s Jan 8 06:52:10.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:12.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 42.352776718s Jan 8 06:52:12.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:14.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 44.353033056s Jan 8 06:52:14.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:16.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 46.353177002s Jan 8 06:52:16.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:18.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 48.352687757s Jan 8 06:52:18.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:20.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 50.353611984s Jan 8 06:52:20.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:22.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 52.353332993s Jan 8 06:52:22.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:24.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 54.353116306s Jan 8 06:52:24.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:26.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 56.353074791s Jan 8 06:52:26.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:28.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 58.353449569s Jan 8 06:52:28.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:30.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.352730713s Jan 8 06:52:30.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:32.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.353364554s Jan 8 06:52:32.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:34.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.353615319s Jan 8 06:52:34.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:36.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.353689787s Jan 8 06:52:36.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:38.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.353699105s Jan 8 06:52:38.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:40.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.353632681s Jan 8 06:52:40.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:42.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.3530353s Jan 8 06:52:42.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:44.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.353106106s Jan 8 06:52:44.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:46.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.352501227s Jan 8 06:52:46.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:48.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.353146522s Jan 8 06:52:48.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:50.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.353273594s Jan 8 06:52:50.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:52.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.352975016s Jan 8 06:52:52.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:54.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.352705092s Jan 8 06:52:54.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:56.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.353678841s Jan 8 06:52:56.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:52:58.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.353261443s Jan 8 06:52:58.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:00.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.353216047s Jan 8 06:53:00.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:02.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.353721651s Jan 8 06:53:02.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:04.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.353830479s Jan 8 06:53:04.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:06.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.354261515s Jan 8 06:53:06.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:08.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.35338213s Jan 8 06:53:08.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:10.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.353646282s Jan 8 06:53:10.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:12.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.353233331s Jan 8 06:53:12.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:14.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.35267868s Jan 8 06:53:14.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:16.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.353674897s Jan 8 06:53:16.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:18.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.353046077s Jan 8 06:53:18.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:20.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.353226824s Jan 8 06:53:20.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:22.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.352595946s Jan 8 06:53:22.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:24.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.35263301s Jan 8 06:53:24.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:26.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.353881483s Jan 8 06:53:26.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:28.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.353260703s Jan 8 06:53:28.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:30.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.352291604s Jan 8 06:53:30.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:32.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.353040273s Jan 8 06:53:32.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:34.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.353194441s Jan 8 06:53:34.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:36.325: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.355295457s Jan 8 06:53:36.325: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:38.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.353463724s Jan 8 06:53:38.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:40.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.353213445s Jan 8 06:53:40.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:42.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.352584924s Jan 8 06:53:42.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:44.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.353244236s Jan 8 06:53:44.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:46.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.353170463s Jan 8 06:53:46.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:48.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.352730823s Jan 8 06:53:48.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:50.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.353548193s Jan 8 06:53:50.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:52.325: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.355184425s Jan 8 06:53:52.325: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:54.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.352538522s Jan 8 06:53:54.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:56.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.352455435s Jan 8 06:53:56.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:53:58.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.352108215s Jan 8 06:53:58.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:00.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.352435723s Jan 8 06:54:00.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:02.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.353191343s Jan 8 06:54:02.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:04.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.353242178s Jan 8 06:54:04.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:06.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.352689045s Jan 8 06:54:06.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:08.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.353346052s Jan 8 06:54:08.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:10.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.352749625s Jan 8 06:54:10.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:12.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.353114572s Jan 8 06:54:12.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:14.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.35280652s Jan 8 06:54:14.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:16.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.353405463s Jan 8 06:54:16.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:18.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.353149911s Jan 8 06:54:18.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:20.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.352466665s Jan 8 06:54:20.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:22.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.352764667s Jan 8 06:54:22.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:24.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.352690858s Jan 8 06:54:24.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:26.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.352544682s Jan 8 06:54:26.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:28.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.353106723s Jan 8 06:54:28.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:30.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.354233548s Jan 8 06:54:30.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:32.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.352754691s Jan 8 06:54:32.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:34.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.352379627s Jan 8 06:54:34.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:36.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.353196697s Jan 8 06:54:36.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:38.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.35217168s Jan 8 06:54:38.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:40.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.353200017s Jan 8 06:54:40.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:42.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.353135351s Jan 8 06:54:42.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:44.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.353701756s Jan 8 06:54:44.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:46.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.353813406s Jan 8 06:54:46.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:48.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.353064809s Jan 8 06:54:48.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:50.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.35241341s Jan 8 06:54:50.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:52.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.353111548s Jan 8 06:54:52.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:54.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.353186471s Jan 8 06:54:54.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:56.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.353028123s Jan 8 06:54:56.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:54:58.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.353097766s Jan 8 06:54:58.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:00.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.352787138s Jan 8 06:55:00.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:02.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.352843431s Jan 8 06:55:02.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:04.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.353539967s Jan 8 06:55:04.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:06.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.35341443s Jan 8 06:55:06.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:08.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.353374256s Jan 8 06:55:08.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:10.324: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.353704801s Jan 8 06:55:10.324: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:12.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.35260277s Jan 8 06:55:12.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:14.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.352154419s Jan 8 06:55:14.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:16.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.353156799s Jan 8 06:55:16.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:18.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.352260446s Jan 8 06:55:18.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:20.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.352916694s Jan 8 06:55:20.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:22.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.352823507s Jan 8 06:55:22.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:24.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.353256138s Jan 8 06:55:24.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:26.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.353293676s Jan 8 06:55:26.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:28.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.352680274s Jan 8 06:55:28.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:30.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.353033615s Jan 8 06:55:30.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:32.325: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.355165503s Jan 8 06:55:32.325: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:34.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.353320303s Jan 8 06:55:34.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:36.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.352485147s Jan 8 06:55:36.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:38.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.353251839s Jan 8 06:55:38.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:40.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.35343183s Jan 8 06:55:40.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:42.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.352325122s Jan 8 06:55:42.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:44.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.35310813s Jan 8 06:55:44.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:46.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.353422428s Jan 8 06:55:46.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:48.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.352886286s Jan 8 06:55:48.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:50.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.352925892s Jan 8 06:55:50.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:52.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.352638697s Jan 8 06:55:52.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:54.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.353022983s Jan 8 06:55:54.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:56.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.352450468s Jan 8 06:55:56.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:55:58.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.35318657s Jan 8 06:55:58.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:00.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.353480981s Jan 8 06:56:00.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:02.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.353537201s Jan 8 06:56:02.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:04.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.353296485s Jan 8 06:56:04.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:06.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.352333816s Jan 8 06:56:06.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:08.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.353498365s Jan 8 06:56:08.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:10.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.352410847s Jan 8 06:56:10.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:12.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.353361411s Jan 8 06:56:12.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:14.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.353023145s Jan 8 06:56:14.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:16.322: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.352410515s Jan 8 06:56:16.322: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:18.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.352952017s Jan 8 06:56:18.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:20.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.353424558s Jan 8 06:56:20.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:22.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.353347077s Jan 8 06:56:22.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:24.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.35335866s Jan 8 06:56:24.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:26.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.352941464s Jan 8 06:56:26.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:28.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.353234503s Jan 8 06:56:28.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:30.323: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.352944889s Jan 8 06:56:30.323: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:30.499: INFO: Pod "busybox1": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.529025654s Jan 8 06:56:30.499: INFO: Error evaluating pod condition running and ready: want pod 'busybox1' on 'i-0685adc783a822be6' to be 'Running' but was 'Pending' Jan 8 06:56:30.499: INFO: Pod busybox1 failed to be running and ready. Jan 8 06:56:30.499: INFO: Wanted all 1 pods to be running and ready. Result: false. Pods: [busybox1] Jan 8 06:56:30.499: FAIL: Expected <bool>: false to equal <bool>: true Full Stack Trace k8s.io/kubernetes/test/e2e/kubectl.glob..func1.22.1() test/e2e/kubectl/kubectl.go:1537 +0x205 [AfterEach] Kubectl copy test/e2e/kubectl/kubectl.go:1539 �[1mSTEP:�[0m using delete to clean up resources �[38;5;243m01/08/23 06:56:30.499�[0m Jan 8 06:56:30.500: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-6574 delete --grace-period=0 --force -f -' Jan 8 06:56:31.336: INFO: stderr: "Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.\n" Jan 8 06:56:31.336: INFO: stdout: "pod \"busybox1\" force deleted\n" Jan 8 06:56:31.336: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-6574 get rc,svc -l app=busybox1 --no-headers' Jan 8 06:56:32.173: INFO: stderr: "No resources found in kubectl-6574 namespace.\n" Jan 8 06:56:32.173: INFO: stdout: "" Jan 8 06:56:32.173: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-6574 get pods -l app=busybox1 -o go-template={{ range .items }}{{ if not .metadata.deletionTimestamp }}{{ .metadata.name }}{{ "\n" }}{{ end }}{{ end }}' Jan 8 06:56:32.818: INFO: stderr: "" Jan 8 06:56:32.818: INFO: stdout: "" [AfterEach] [sig-cli] Kubectl client test/e2e/framework/node/init/init.go:32 Jan 8 06:56:32.818: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-cli] Kubectl client test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-cli] Kubectl client dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/08/23 06:56:32.996�[0m �[1mSTEP:�[0m Collecting events from namespace "kubectl-6574". �[38;5;243m01/08/23 06:56:32.996�[0m �[1mSTEP:�[0m Found 11 events. �[38;5;243m01/08/23 06:56:33.172�[0m Jan 8 06:56:33.173: INFO: At 2023-01-08 06:51:29 +0000 UTC - event for busybox1: {default-scheduler } Scheduled: Successfully assigned kubectl-6574/busybox1 to i-0685adc783a822be6 Jan 8 06:56:33.173: INFO: At 2023-01-08 06:51:30 +0000 UTC - event for busybox1: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5a19249fa4f445e9a0bfed4054f371cd393a2d9b48a206605e38e52a349e4cef": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:56:33.173: INFO: At 2023-01-08 06:51:44 +0000 UTC - event for busybox1: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "35172a6ba2722a11f67b27cb32026cb1713bc8406f16f5a519d7268af0cda3c6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:56:33.173: INFO: At 2023-01-08 06:51:59 +0000 UTC - event for busybox1: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4da362433aada473cf80b5d81958ea0fb951ca27180de1e1a6103188e508bbd5": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:56:33.173: INFO: At 2023-01-08 06:52:11 +0000 UTC - event for busybox1: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "19c1337f6458930682e107c969af4924bf1dc0b158d86503fa86955d7f7ff9cf": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:56:33.173: INFO: At 2023-01-08 06:52:22 +0000 UTC - event for busybox1: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1904a33574be8138c33e4daf8c3f093ce40e41ad3a5b65255a9ac5ad83e01103": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:56:33.173: INFO: At 2023-01-08 06:52:36 +0000 UTC - event for busybox1: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d340dc98ebcc360a98190ddd7305234efab6a4d03b0ae8b092ea6f652c037cf4": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:56:33.173: INFO: At 2023-01-08 06:52:49 +0000 UTC - event for busybox1: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3d6e22e5c57f1862a7b6010128566fdc71ee7f4f95f06497bf0bfd5b8aeac3f5": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:56:33.173: INFO: At 2023-01-08 06:53:03 +0000 UTC - event for busybox1: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8777b114e4d3645a152a705d5bac0052476fe36cd39c76a8718974c96a217c28": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:56:33.173: INFO: At 2023-01-08 06:53:16 +0000 UTC - event for busybox1: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0a49eddb00ed68f0517e87fcd9cac8105e0bfd0ddfaa0edc08843f624fe60d5a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:56:33.173: INFO: At 2023-01-08 06:53:30 +0000 UTC - event for busybox1: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7e4a1aac8adc046994b72aa8e76569b34735adea6eec3406376ff83cc8521231": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:56:33.348: INFO: POD NODE PHASE GRACE CONDITIONS Jan 8 06:56:33.348: INFO: Jan 8 06:56:33.526: INFO: Logging node info for node i-0685adc783a822be6 Jan 8 06:56:33.702: INFO: Node Info: &Node{ObjectMeta:{i-0685adc783a822be6 3eb588c7-cc76-44b4-b28a-492b9989254b 43426 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0685adc783a822be6 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0685adc783a822be6 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.112 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-8725":"i-0685adc783a822be6","ebs.csi.aws.com":"i-0685adc783a822be6"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:55:45 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-08 06:55:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0685adc783a822be6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:45 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:45 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:45 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:55:45 +0000 UTC,LastTransitionTime:2023-01-08 06:25:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.112,},NodeAddress{Type:ExternalIP,Address:43.201.46.149,},NodeAddress{Type:InternalDNS,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-201-46-149.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dccafbf198926b990228825ae0608,SystemUUID:ec2dccaf-bf19-8926-b990-228825ae0608,BootID:b8ff627c-af94-4730-be1d-6c7bfbfc818f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0488b0c43122e11c1 kubernetes.io/csi/ebs.csi.aws.com^vol-0b8657a4eb6501329],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0b8657a4eb6501329,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0488b0c43122e11c1,DevicePath:,},},Config:nil,},} Jan 8 06:56:33.703: INFO: Logging kubelet events for node i-0685adc783a822be6 Jan 8 06:56:33.881: INFO: Logging pods the kubelet thinks is on node i-0685adc783a822be6 Jan 8 06:56:34.067: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-zgmck started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:34.067: INFO: inline-volume-tester-9qdkl started at 2023-01-08 06:52:11 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 8 06:56:34.067: INFO: csi-mockplugin-0 started at 2023-01-08 06:50:39 +0000 UTC (0+3 container statuses recorded) Jan 8 06:56:34.067: INFO: Container csi-provisioner ready: true, restart count 0 Jan 8 06:56:34.067: INFO: Container driver-registrar ready: true, restart count 0 Jan 8 06:56:34.067: INFO: Container mock ready: true, restart count 0 Jan 8 06:56:34.067: INFO: cilium-9vhbx started at 2023-01-08 06:25:38 +0000 UTC (1+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:56:34.067: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:56:34.067: INFO: pod-disruption-failure-ignore-1-kfqqr started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container c ready: false, restart count 0 Jan 8 06:56:34.067: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-gq5pr started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:34.067: INFO: service-proxy-toggled-dh67n started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container service-proxy-toggled ready: false, restart count 0 Jan 8 06:56:34.067: INFO: liveness-51325787-aaa2-4ac1-a3f9-8ff47e3fa793 started at 2023-01-08 06:54:56 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container agnhost-container ready: false, restart count 0 Jan 8 06:56:34.067: INFO: pod-subpath-test-inlinevolume-plpm started at 2023-01-08 06:52:37 +0000 UTC (2+2 container statuses recorded) Jan 8 06:56:34.067: INFO: Init container init-volume-inlinevolume-plpm ready: false, restart count 0 Jan 8 06:56:34.067: INFO: Init container test-init-subpath-inlinevolume-plpm ready: false, restart count 0 Jan 8 06:56:34.067: INFO: Container test-container-subpath-inlinevolume-plpm ready: false, restart count 0 Jan 8 06:56:34.067: INFO: Container test-container-volume-inlinevolume-plpm ready: false, restart count 0 Jan 8 06:56:34.067: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:51:13 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container csi-attacher ready: false, restart count 0 Jan 8 06:56:34.067: INFO: pod-937b6812-ddb0-47fe-af8b-07288c742e73 started at 2023-01-08 06:52:10 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container write-pod ready: false, restart count 0 Jan 8 06:56:34.067: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-svbp2 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:34.067: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mpw58 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:34.067: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rrmzm started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:34.067: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:50:39 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container csi-attacher ready: false, restart count 0 Jan 8 06:56:34.067: INFO: pod-disruption-failure-ignore-0-w4rvn started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container c ready: false, restart count 0 Jan 8 06:56:34.067: INFO: pod-projected-secrets-907df958-d79f-42f9-87ee-41301a6f40be started at 2023-01-08 06:55:13 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container secret-volume-test ready: false, restart count 0 Jan 8 06:56:34.067: INFO: ebs-csi-node-ft7bw started at 2023-01-08 06:25:38 +0000 UTC (0+3 container statuses recorded) Jan 8 06:56:34.067: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:56:34.067: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:56:34.067: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:56:34.067: INFO: suspend-false-to-true-97wd6 started at 2023-01-08 06:53:18 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container c ready: false, restart count 0 Jan 8 06:56:34.067: INFO: csi-mockplugin-0 started at 2023-01-08 06:56:04 +0000 UTC (0+4 container statuses recorded) Jan 8 06:56:34.067: INFO: Container busybox ready: false, restart count 0 Jan 8 06:56:34.067: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 06:56:34.067: INFO: Container driver-registrar ready: false, restart count 0 Jan 8 06:56:34.067: INFO: Container mock ready: false, restart count 0 Jan 8 06:56:34.067: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-q97zd started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:34.067: INFO: affinity-clusterip-zxklh started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container affinity-clusterip ready: false, restart count 0 Jan 8 06:56:34.067: INFO: service-proxy-disabled-6pp8x started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container service-proxy-disabled ready: true, restart count 0 Jan 8 06:56:34.067: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2bn4h started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:34.067: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v9tjx started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:34.067: INFO: csi-mockplugin-0 started at 2023-01-08 06:51:13 +0000 UTC (0+3 container statuses recorded) Jan 8 06:56:34.067: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 06:56:34.067: INFO: Container driver-registrar ready: false, restart count 0 Jan 8 06:56:34.067: INFO: Container mock ready: false, restart count 0 Jan 8 06:56:34.067: INFO: agnhost-primary-9vrbd started at 2023-01-08 06:54:54 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container agnhost-primary ready: false, restart count 0 Jan 8 06:56:34.067: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-97qp6 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:34.067: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-4d46c started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:34.067: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:34.689: INFO: Latency metrics for node i-0685adc783a822be6 Jan 8 06:56:34.689: INFO: Logging node info for node i-0bd196cab651da520 Jan 8 06:56:34.865: INFO: Node Info: &Node{ObjectMeta:{i-0bd196cab651da520 f7f00780-d24e-4c8e-b54d-373598e1d1d2 42453 0 2023-01-08 06:26:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0bd196cab651da520 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0bd196cab651da520 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.104 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bd196cab651da520"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:26:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:52:47 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-08 06:52:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0bd196cab651da520,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:52:46 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:52:46 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:52:46 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:52:46 +0000 UTC,LastTransitionTime:2023-01-08 06:26:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.104,},NodeAddress{Type:ExternalIP,Address:54.180.1.55,},NodeAddress{Type:InternalDNS,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-180-1-55.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec218f9d32a84b2feeb6301255fd576c,SystemUUID:ec218f9d-32a8-4b2f-eeb6-301255fd576c,BootID:accdf7fd-1195-4765-b28e-29d44bf47bac,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-06c6e766891904b0f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-06c6e766891904b0f,DevicePath:,},},Config:nil,},} Jan 8 06:56:34.866: INFO: Logging kubelet events for node i-0bd196cab651da520 Jan 8 06:56:35.044: INFO: Logging pods the kubelet thinks is on node i-0bd196cab651da520 Jan 8 06:56:35.232: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9mp69 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:35.232: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-cddx5 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:35.232: INFO: pod-projected-configmaps-143d6d2e-7b9b-47e8-97c7-c2b8adc28191 started at 2023-01-08 06:52:29 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container agnhost-container ready: false, restart count 0 Jan 8 06:56:35.232: INFO: suspend-false-to-true-xwpqd started at 2023-01-08 06:53:18 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container c ready: false, restart count 0 Jan 8 06:56:35.232: INFO: pod6b22b1b4-d027-4092-9a22-c1f06499be00 started at 2023-01-08 06:56:02 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container nginx ready: false, restart count 0 Jan 8 06:56:35.232: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-6df8l started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:35.232: INFO: busybox-scheduling-0abca61c-e823-4136-8618-ce7deb358c0b started at 2023-01-08 06:55:28 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container busybox-scheduling-0abca61c-e823-4136-8618-ce7deb358c0b ready: false, restart count 0 Jan 8 06:56:35.232: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-vrcjq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:35.232: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9776c started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:35.232: INFO: cilium-s6shx started at 2023-01-08 06:26:02 +0000 UTC (1+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:56:35.232: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:56:35.232: INFO: ebs-csi-node-6grlm started at 2023-01-08 06:26:02 +0000 UTC (0+3 container statuses recorded) Jan 8 06:56:35.232: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:56:35.232: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:56:35.232: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:56:35.232: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9wskq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:35.232: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mphgx started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:35.232: INFO: pod1 started at 2023-01-08 06:51:10 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container agnhost-container ready: true, restart count 0 Jan 8 06:56:35.232: INFO: affinity-clusterip-mrptn started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container affinity-clusterip ready: false, restart count 0 Jan 8 06:56:35.232: INFO: service-proxy-toggled-8nfkz started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container service-proxy-toggled ready: false, restart count 0 Jan 8 06:56:35.232: INFO: execpodgmcm4 started at 2023-01-08 06:55:35 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container agnhost-container ready: false, restart count 0 Jan 8 06:56:35.232: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-76nxw started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:35.232: INFO: sample-webhook-deployment-865554f4d9-4kppl started at 2023-01-08 06:55:41 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container sample-webhook ready: false, restart count 0 Jan 8 06:56:35.232: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v44bk started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:35.232: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-75r8w started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:35.232: INFO: pod-a15c4150-c88e-49b1-a111-7f2271cf819c started at 2023-01-08 06:46:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:35.232: INFO: Container write-pod ready: true, restart count 0 Jan 8 06:56:35.927: INFO: Latency metrics for node i-0bd196cab651da520 Jan 8 06:56:35.927: INFO: Logging node info for node i-0d149644e2edcf9cf Jan 8 06:56:36.103: INFO: Node Info: &Node{ObjectMeta:{i-0d149644e2edcf9cf b448ef72-6e61-4ad5-aa29-d63cc0f94945 43645 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0d149644e2edcf9cf kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0d149644e2edcf9cf topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.115 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0d149644e2edcf9cf"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:56:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0d149644e2edcf9cf,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.115,},NodeAddress{Type:ExternalIP,Address:3.35.219.218,},NodeAddress{Type:InternalDNS,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-35-219-218.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec26c6a9af66729e278545162595eb2d,SystemUUID:ec26c6a9-af66-729e-2785-45162595eb2d,BootID:959da9e3-fb86-442a-958f-2a49c509cdf4,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 8 06:56:36.104: INFO: Logging kubelet events for node i-0d149644e2edcf9cf Jan 8 06:56:36.282: INFO: Logging pods the kubelet thinks is on node i-0d149644e2edcf9cf Jan 8 06:56:36.471: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-xrmq4 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:36.471: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-7gwsq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:36.471: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nsfnm started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:36.471: INFO: coredns-autoscaler-7cb5c5b969-xqnk5 started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Container autoscaler ready: true, restart count 0 Jan 8 06:56:36.471: INFO: service-proxy-disabled-v5f82 started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Container service-proxy-disabled ready: true, restart count 0 Jan 8 06:56:36.471: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-tgn6r started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:36.471: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:52:13 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Container csi-attacher ready: false, restart count 0 Jan 8 06:56:36.471: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9b2xq started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:36.471: INFO: probe-test-8a476652-ee98-415a-b6f6-009d080c5f05 started at 2023-01-08 06:51:44 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Container probe-test-8a476652-ee98-415a-b6f6-009d080c5f05 ready: false, restart count 0 Jan 8 06:56:36.471: INFO: csi-mockplugin-0 started at 2023-01-08 06:52:13 +0000 UTC (0+3 container statuses recorded) Jan 8 06:56:36.471: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 06:56:36.471: INFO: Container driver-registrar ready: false, restart count 0 Jan 8 06:56:36.471: INFO: Container mock ready: false, restart count 0 Jan 8 06:56:36.471: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-njh2f started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:36.471: INFO: cilium-58twl started at 2023-01-08 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:56:36.471: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:56:36.471: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rv5qq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:36.471: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-flhps started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:36.471: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2vh6v started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:36.471: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nt9jr started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0 Jan 8 06:56:36.471: INFO: ebs-csi-node-lfcwb started at 2023-01-08 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 8 06:56:36.471: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:56:36.471: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:56:36.471: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:56:36.471: INFO: coredns-559769c974-9q4qg started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:36.471: INFO: Container coredns ready: true, restart count 0 Jan 8 06:56:36.471: INFO: csi-hostpathplugin-0 started at 2023-01-08 06:52:05 +0000 UTC (0+7 container statuses recorded) Jan 8 06:56:36.471: INFO: Container csi-attacher ready: false, restart count 0 Jan 8 06:56:36.471: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 06:56:36.471: INFO: Container csi-resizer ready: false, restart count 0 Jan 8 06:56:36.471: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 8 06:56:36.471: INFO: Container hostpath ready: false, restart count 0 Jan 8 06:56:36.471: INFO: Container liveness-probe ready: false, restart count 0 Jan 8 06:56:36.471: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 8 06:56:37.100: INFO: Latency metrics for node i-0d149644e2edcf9cf Jan 8 06:56:37.100: INFO: Logging node info for node i-0eee6113fd9f81d94 Jan 8 06:56:37.276: INFO: Node Info: &Node{ObjectMeta:{i-0eee6113fd9f81d94 6de592fc-f11c-4ba3-afa0-707bb4ca29b2 41943 0 2023-01-08 06:23:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0eee6113fd9f81d94 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.48.64 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eee6113fd9f81d94"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-08 06:23:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-08 06:24:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-08 06:24:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-08 06:51:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0eee6113fd9f81d94,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:51:55 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:51:55 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:51:55 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:51:55 +0000 UTC,LastTransitionTime:2023-01-08 06:24:35 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.48.64,},NodeAddress{Type:ExternalIP,Address:3.35.206.95,},NodeAddress{Type:InternalDNS,Address:i-0eee6113fd9f81d94.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0eee6113fd9f81d94.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-35-206-95.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d8cf21861b15bb593210ff59f6044,SystemUUID:ec2d8cf2-1861-b15b-b593-210ff59f6044,BootID:fcad947c-a63f-4722-9a63-22eac74bea9f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43453935,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42803460,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967354,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 8 06:56:37.277: INFO: Logging kubelet events for node i-0eee6113fd9f81d94 Jan 8 06:56:37.455: INFO: Logging pods the kubelet thinks is on node i-0eee6113fd9f81d94 Jan 8 06:56:37.636: INFO: dns-controller-69987775c6-v4cmg started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:37.636: INFO: Container dns-controller ready: true, restart count 0 Jan 8 06:56:37.636: INFO: kube-scheduler-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:37.636: INFO: Container kube-scheduler ready: true, restart count 0 Jan 8 06:56:37.636: INFO: kube-apiserver-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+2 container statuses recorded) Jan 8 06:56:37.636: INFO: Container healthcheck ready: true, restart count 0 Jan 8 06:56:37.636: INFO: Container kube-apiserver ready: true, restart count 1 Jan 8 06:56:37.636: INFO: kube-controller-manager-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:37.636: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 8 06:56:37.636: INFO: ebs-csi-node-nspqr started at 2023-01-08 06:24:21 +0000 UTC (0+3 container statuses recorded) Jan 8 06:56:37.636: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:56:37.636: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:56:37.636: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:56:37.636: INFO: aws-cloud-controller-manager-d8d56 started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:37.636: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 8 06:56:37.636: INFO: cilium-operator-5dd44dc49f-fd8vl started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:37.636: INFO: Container cilium-operator ready: true, restart count 0 Jan 8 06:56:37.636: INFO: ebs-csi-controller-89b649c94-vzzj9 started at 2023-01-08 06:24:21 +0000 UTC (0+5 container statuses recorded) Jan 8 06:56:37.636: INFO: Container csi-attacher ready: true, restart count 0 Jan 8 06:56:37.636: INFO: Container csi-provisioner ready: true, restart count 0 Jan 8 06:56:37.636: INFO: Container csi-resizer ready: true, restart count 0 Jan 8 06:56:37.636: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:56:37.636: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:56:37.636: INFO: etcd-manager-events-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:37.636: INFO: Container etcd-manager ready: true, restart count 0 Jan 8 06:56:37.636: INFO: etcd-manager-main-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:37.636: INFO: Container etcd-manager ready: true, restart count 0 Jan 8 06:56:37.636: INFO: kops-controller-242qq started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:37.636: INFO: Container kops-controller ready: true, restart count 0 Jan 8 06:56:37.636: INFO: cilium-f22px started at 2023-01-08 06:24:21 +0000 UTC (1+1 container statuses recorded) Jan 8 06:56:37.636: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:56:37.636: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:56:38.235: INFO: Latency metrics for node i-0eee6113fd9f81d94 Jan 8 06:56:38.235: INFO: Logging node info for node i-0f469f08be563dad4 Jan 8 06:56:38.411: INFO: Node Info: &Node{ObjectMeta:{i-0f469f08be563dad4 fddb222c-a6ac-415f-83dc-4b71acb0e5ab 43138 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0f469f08be563dad4 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0f469f08be563dad4 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.57.173 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0f469f08be563dad4"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:55:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0f469f08be563dad4,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.57.173,},NodeAddress{Type:ExternalIP,Address:3.38.152.241,},NodeAddress{Type:InternalDNS,Address:i-0f469f08be563dad4.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f469f08be563dad4.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-38-152-241.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2abc7b1d816532cfb4787d85cb98ea,SystemUUID:ec2abc7b-1d81-6532-cfb4-787d85cb98ea,BootID:442daa6f-46d1-4c32-9182-8b6204c22aba,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 8 06:56:38.412: INFO: Logging kubelet events for node i-0f469f08be563dad4 Jan 8 06:56:38.590: INFO: Logging pods the kubelet thinks is on node i-0f469f08be563dad4 Jan 8 06:56:38.775: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-b6m8x started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:38.775: INFO: affinity-clusterip-8kclb started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Container affinity-clusterip ready: false, restart count 0 Jan 8 06:56:38.775: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-dvpgx started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:38.775: INFO: cilium-24rbm started at 2023-01-08 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:56:38.775: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:56:38.775: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-ktkbb started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:38.775: INFO: ebs-csi-node-9s8qc started at 2023-01-08 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 8 06:56:38.775: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:56:38.775: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:56:38.775: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:56:38.775: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-x88cb started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:38.775: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-h5c5c started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:38.775: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-jc58p started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:38.775: INFO: service-proxy-disabled-bqbxv started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Container service-proxy-disabled ready: true, restart count 0 Jan 8 06:56:38.775: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-gdlbp started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:38.775: INFO: service-proxy-toggled-7qcjj started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Container service-proxy-toggled ready: true, restart count 0 Jan 8 06:56:38.775: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-cwg4t started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:38.775: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-57rwf started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0 Jan 8 06:56:38.775: INFO: pod-configmaps-393a999b-3a18-4df6-95b4-26106a4b0e61 started at 2023-01-08 06:51:56 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Container agnhost-container ready: false, restart count 0 Jan 8 06:56:38.775: INFO: coredns-559769c974-f9s4x started at 2023-01-08 06:26:19 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Container coredns ready: true, restart count 0 Jan 8 06:56:38.775: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-glvzd started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:56:38.775: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:56:39.430: INFO: Latency metrics for node i-0f469f08be563dad4 [DeferCleanup (Each)] [sig-cli] Kubectl client tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "kubectl-6574" for this suite. �[38;5;243m01/08/23 06:56:39.43�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sclient\sKubectl\sexpose\sshould\screate\sservices\sfor\src\s\s\[Conformance\]$'
test/e2e/kubectl/kubectl.go:288 k8s.io/kubernetes/test/e2e/kubectl.glob..func1.4(0x1) test/e2e/kubectl/kubectl.go:288 +0x13b k8s.io/kubernetes/test/e2e/kubectl.glob..func1.20.1() test/e2e/kubectl/kubectl.go:1427 +0x1b4from junit_01.xml
[BeforeEach] [sig-cli] Kubectl client set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/08/23 06:54:50.95�[0m Jan 8 06:54:50.950: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename kubectl �[38;5;243m01/08/23 06:54:50.952�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/08/23 06:54:51.483�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/08/23 06:54:51.834�[0m [BeforeEach] [sig-cli] Kubectl client test/e2e/framework/metrics/init/init.go:31 [BeforeEach] [sig-cli] Kubectl client test/e2e/kubectl/kubectl.go:274 [It] should create services for rc [Conformance] test/e2e/kubectl/kubectl.go:1415 �[1mSTEP:�[0m creating Agnhost RC �[38;5;243m01/08/23 06:54:52.186�[0m Jan 8 06:54:52.186: INFO: namespace kubectl-4507 Jan 8 06:54:52.186: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/60c1a7e1-8f1c-11ed-84f7-72cdabe2d9ae/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-4507 create -f -' Jan 8 06:54:54.267: INFO: stderr: "" Jan 8 06:54:54.267: INFO: stdout: "replicationcontroller/agnhost-primary created\n" �[1mSTEP:�[0m Waiting for Agnhost primary to start. �[38;5;243m01/08/23 06:54:54.267�[0m Jan 8 06:54:55.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:54:55.444: INFO: Found 0 / 1 Jan 8 06:54:56.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:54:56.445: INFO: Found 0 / 1 Jan 8 06:54:57.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:54:57.445: INFO: Found 0 / 1 Jan 8 06:54:58.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:54:58.444: INFO: Found 0 / 1 Jan 8 06:54:59.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:54:59.445: INFO: Found 0 / 1 Jan 8 06:55:00.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:00.445: INFO: Found 0 / 1 Jan 8 06:55:01.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:01.445: INFO: Found 0 / 1 Jan 8 06:55:02.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:02.445: INFO: Found 0 / 1 Jan 8 06:55:03.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:03.445: INFO: Found 0 / 1 Jan 8 06:55:04.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:04.445: INFO: Found 0 / 1 Jan 8 06:55:05.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:05.446: INFO: Found 0 / 1 Jan 8 06:55:06.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:06.444: INFO: Found 0 / 1 Jan 8 06:55:07.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:07.445: INFO: Found 0 / 1 Jan 8 06:55:08.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:08.444: INFO: Found 0 / 1 Jan 8 06:55:09.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:09.446: INFO: Found 0 / 1 Jan 8 06:55:10.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:10.445: INFO: Found 0 / 1 Jan 8 06:55:11.450: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:11.450: INFO: Found 0 / 1 Jan 8 06:55:12.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:12.445: INFO: Found 0 / 1 Jan 8 06:55:13.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:13.445: INFO: Found 0 / 1 Jan 8 06:55:14.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:14.445: INFO: Found 0 / 1 Jan 8 06:55:15.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:15.446: INFO: Found 0 / 1 Jan 8 06:55:16.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:16.445: INFO: Found 0 / 1 Jan 8 06:55:17.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:17.444: INFO: Found 0 / 1 Jan 8 06:55:18.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:18.445: INFO: Found 0 / 1 Jan 8 06:55:19.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:19.444: INFO: Found 0 / 1 Jan 8 06:55:20.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:20.445: INFO: Found 0 / 1 Jan 8 06:55:21.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:21.445: INFO: Found 0 / 1 Jan 8 06:55:22.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:22.445: INFO: Found 0 / 1 Jan 8 06:55:23.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:23.444: INFO: Found 0 / 1 Jan 8 06:55:24.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:24.445: INFO: Found 0 / 1 Jan 8 06:55:25.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:25.446: INFO: Found 0 / 1 Jan 8 06:55:26.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:26.445: INFO: Found 0 / 1 Jan 8 06:55:27.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:27.445: INFO: Found 0 / 1 Jan 8 06:55:28.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:28.445: INFO: Found 0 / 1 Jan 8 06:55:29.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:29.444: INFO: Found 0 / 1 Jan 8 06:55:30.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:30.446: INFO: Found 0 / 1 Jan 8 06:55:31.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:31.444: INFO: Found 0 / 1 Jan 8 06:55:32.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:32.445: INFO: Found 0 / 1 Jan 8 06:55:33.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:33.445: INFO: Found 0 / 1 Jan 8 06:55:34.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:34.445: INFO: Found 0 / 1 Jan 8 06:55:35.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:35.445: INFO: Found 0 / 1 Jan 8 06:55:36.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:36.445: INFO: Found 0 / 1 Jan 8 06:55:37.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:37.445: INFO: Found 0 / 1 Jan 8 06:55:38.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:38.445: INFO: Found 0 / 1 Jan 8 06:55:39.456: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:39.456: INFO: Found 0 / 1 Jan 8 06:55:40.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:40.445: INFO: Found 0 / 1 Jan 8 06:55:41.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:41.445: INFO: Found 0 / 1 Jan 8 06:55:42.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:42.445: INFO: Found 0 / 1 Jan 8 06:55:43.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:43.445: INFO: Found 0 / 1 Jan 8 06:55:44.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:44.445: INFO: Found 0 / 1 Jan 8 06:55:45.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:45.445: INFO: Found 0 / 1 Jan 8 06:55:46.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:46.445: INFO: Found 0 / 1 Jan 8 06:55:47.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:47.445: INFO: Found 0 / 1 Jan 8 06:55:48.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:48.444: INFO: Found 0 / 1 Jan 8 06:55:49.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:49.445: INFO: Found 0 / 1 Jan 8 06:55:50.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:50.445: INFO: Found 0 / 1 Jan 8 06:55:51.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:51.445: INFO: Found 0 / 1 Jan 8 06:55:52.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:52.445: INFO: Found 0 / 1 Jan 8 06:55:53.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:53.446: INFO: Found 0 / 1 Jan 8 06:55:54.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:54.444: INFO: Found 0 / 1 Jan 8 06:55:55.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:55.445: INFO: Found 0 / 1 Jan 8 06:55:56.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:56.445: INFO: Found 0 / 1 Jan 8 06:55:57.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:57.445: INFO: Found 0 / 1 Jan 8 06:55:58.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:58.445: INFO: Found 0 / 1 Jan 8 06:55:59.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:55:59.445: INFO: Found 0 / 1 Jan 8 06:56:00.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:00.445: INFO: Found 0 / 1 Jan 8 06:56:01.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:01.445: INFO: Found 0 / 1 Jan 8 06:56:02.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:02.445: INFO: Found 0 / 1 Jan 8 06:56:03.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:03.445: INFO: Found 0 / 1 Jan 8 06:56:04.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:04.445: INFO: Found 0 / 1 Jan 8 06:56:05.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:05.446: INFO: Found 0 / 1 Jan 8 06:56:06.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:06.445: INFO: Found 0 / 1 Jan 8 06:56:07.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:07.445: INFO: Found 0 / 1 Jan 8 06:56:08.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:08.445: INFO: Found 0 / 1 Jan 8 06:56:09.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:09.445: INFO: Found 0 / 1 Jan 8 06:56:10.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:10.445: INFO: Found 0 / 1 Jan 8 06:56:11.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:11.445: INFO: Found 0 / 1 Jan 8 06:56:12.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:12.445: INFO: Found 0 / 1 Jan 8 06:56:13.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:13.445: INFO: Found 0 / 1 Jan 8 06:56:14.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:14.445: INFO: Found 0 / 1 Jan 8 06:56:15.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:15.445: INFO: Found 0 / 1 Jan 8 06:56:16.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:16.445: INFO: Found 0 / 1 Jan 8 06:56:17.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:17.445: INFO: Found 0 / 1 Jan 8 06:56:18.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:18.445: INFO: Found 0 / 1 Jan 8 06:56:19.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:19.445: INFO: Found 0 / 1 Jan 8 06:56:20.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:20.444: INFO: Found 0 / 1 Jan 8 06:56:21.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:21.445: INFO: Found 0 / 1 Jan 8 06:56:22.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:22.446: INFO: Found 0 / 1 Jan 8 06:56:23.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:23.445: INFO: Found 0 / 1 Jan 8 06:56:24.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:24.444: INFO: Found 0 / 1 Jan 8 06:56:25.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:25.445: INFO: Found 0 / 1 Jan 8 06:56:26.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:26.445: INFO: Found 0 / 1 Jan 8 06:56:27.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:27.445: INFO: Found 0 / 1 Jan 8 06:56:28.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:28.445: INFO: Found 0 / 1 Jan 8 06:56:29.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:29.445: INFO: Found 0 / 1 Jan 8 06:56:30.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:30.445: INFO: Found 0 / 1 Jan 8 06:56:31.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:31.445: INFO: Found 0 / 1 Jan 8 06:56:32.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:32.444: INFO: Found 0 / 1 Jan 8 06:56:33.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:33.445: INFO: Found 0 / 1 Jan 8 06:56:34.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:34.445: INFO: Found 0 / 1 Jan 8 06:56:35.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:35.445: INFO: Found 0 / 1 Jan 8 06:56:36.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:36.445: INFO: Found 0 / 1 Jan 8 06:56:37.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:37.445: INFO: Found 0 / 1 Jan 8 06:56:38.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:38.445: INFO: Found 0 / 1 Jan 8 06:56:39.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:39.446: INFO: Found 0 / 1 Jan 8 06:56:40.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:40.444: INFO: Found 0 / 1 Jan 8 06:56:41.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:41.445: INFO: Found 0 / 1 Jan 8 06:56:42.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:42.445: INFO: Found 0 / 1 Jan 8 06:56:43.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:43.445: INFO: Found 0 / 1 Jan 8 06:56:44.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:44.445: INFO: Found 0 / 1 Jan 8 06:56:45.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:45.445: INFO: Found 0 / 1 Jan 8 06:56:46.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:46.445: INFO: Found 0 / 1 Jan 8 06:56:47.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:47.445: INFO: Found 0 / 1 Jan 8 06:56:48.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:48.445: INFO: Found 0 / 1 Jan 8 06:56:49.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:49.445: INFO: Found 0 / 1 Jan 8 06:56:50.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:50.446: INFO: Found 0 / 1 Jan 8 06:56:51.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:51.445: INFO: Found 0 / 1 Jan 8 06:56:52.447: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:52.447: INFO: Found 0 / 1 Jan 8 06:56:53.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:53.445: INFO: Found 0 / 1 Jan 8 06:56:54.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:54.445: INFO: Found 0 / 1 Jan 8 06:56:55.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:55.445: INFO: Found 0 / 1 Jan 8 06:56:56.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:56.445: INFO: Found 0 / 1 Jan 8 06:56:57.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:57.445: INFO: Found 0 / 1 Jan 8 06:56:58.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:58.445: INFO: Found 0 / 1 Jan 8 06:56:59.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:56:59.445: INFO: Found 0 / 1 Jan 8 06:57:00.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:00.446: INFO: Found 0 / 1 Jan 8 06:57:01.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:01.445: INFO: Found 0 / 1 Jan 8 06:57:02.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:02.444: INFO: Found 0 / 1 Jan 8 06:57:03.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:03.444: INFO: Found 0 / 1 Jan 8 06:57:04.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:04.446: INFO: Found 0 / 1 Jan 8 06:57:05.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:05.445: INFO: Found 0 / 1 Jan 8 06:57:06.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:06.445: INFO: Found 0 / 1 Jan 8 06:57:07.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:07.446: INFO: Found 0 / 1 Jan 8 06:57:08.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:08.445: INFO: Found 0 / 1 Jan 8 06:57:09.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:09.445: INFO: Found 0 / 1 Jan 8 06:57:10.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:10.445: INFO: Found 0 / 1 Jan 8 06:57:11.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:11.445: INFO: Found 0 / 1 Jan 8 06:57:12.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:12.445: INFO: Found 0 / 1 Jan 8 06:57:13.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:13.445: INFO: Found 0 / 1 Jan 8 06:57:14.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:14.444: INFO: Found 0 / 1 Jan 8 06:57:15.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:15.445: INFO: Found 0 / 1 Jan 8 06:57:16.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:16.445: INFO: Found 0 / 1 Jan 8 06:57:17.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:17.445: INFO: Found 0 / 1 Jan 8 06:57:18.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:18.445: INFO: Found 0 / 1 Jan 8 06:57:19.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:19.444: INFO: Found 0 / 1 Jan 8 06:57:20.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:20.445: INFO: Found 0 / 1 Jan 8 06:57:21.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:21.445: INFO: Found 0 / 1 Jan 8 06:57:22.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:22.444: INFO: Found 0 / 1 Jan 8 06:57:23.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:23.446: INFO: Found 0 / 1 Jan 8 06:57:24.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:24.445: INFO: Found 0 / 1 Jan 8 06:57:25.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:25.445: INFO: Found 0 / 1 Jan 8 06:57:26.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:26.445: INFO: Found 0 / 1 Jan 8 06:57:27.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:27.445: INFO: Found 0 / 1 Jan 8 06:57:28.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:28.444: INFO: Found 0 / 1 Jan 8 06:57:29.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:29.445: INFO: Found 0 / 1 Jan 8 06:57:30.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:30.445: INFO: Found 0 / 1 Jan 8 06:57:31.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:31.445: INFO: Found 0 / 1 Jan 8 06:57:32.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:32.445: INFO: Found 0 / 1 Jan 8 06:57:33.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:33.446: INFO: Found 0 / 1 Jan 8 06:57:34.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:34.444: INFO: Found 0 / 1 Jan 8 06:57:35.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:35.445: INFO: Found 0 / 1 Jan 8 06:57:36.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:36.445: INFO: Found 0 / 1 Jan 8 06:57:37.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:37.445: INFO: Found 0 / 1 Jan 8 06:57:38.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:38.445: INFO: Found 0 / 1 Jan 8 06:57:39.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:39.444: INFO: Found 0 / 1 Jan 8 06:57:40.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:40.444: INFO: Found 0 / 1 Jan 8 06:57:41.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:41.445: INFO: Found 0 / 1 Jan 8 06:57:42.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:42.445: INFO: Found 0 / 1 Jan 8 06:57:43.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:43.445: INFO: Found 0 / 1 Jan 8 06:57:44.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:44.446: INFO: Found 0 / 1 Jan 8 06:57:45.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:45.445: INFO: Found 0 / 1 Jan 8 06:57:46.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:46.445: INFO: Found 0 / 1 Jan 8 06:57:47.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:47.444: INFO: Found 0 / 1 Jan 8 06:57:48.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:48.445: INFO: Found 0 / 1 Jan 8 06:57:49.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:49.444: INFO: Found 0 / 1 Jan 8 06:57:50.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:50.445: INFO: Found 0 / 1 Jan 8 06:57:51.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:51.445: INFO: Found 0 / 1 Jan 8 06:57:52.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:52.445: INFO: Found 0 / 1 Jan 8 06:57:53.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:53.446: INFO: Found 0 / 1 Jan 8 06:57:54.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:54.445: INFO: Found 0 / 1 Jan 8 06:57:55.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:55.445: INFO: Found 0 / 1 Jan 8 06:57:56.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:56.445: INFO: Found 0 / 1 Jan 8 06:57:57.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:57.445: INFO: Found 0 / 1 Jan 8 06:57:58.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:58.445: INFO: Found 0 / 1 Jan 8 06:57:59.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:57:59.445: INFO: Found 0 / 1 Jan 8 06:58:00.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:00.445: INFO: Found 0 / 1 Jan 8 06:58:01.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:01.445: INFO: Found 0 / 1 Jan 8 06:58:02.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:02.446: INFO: Found 0 / 1 Jan 8 06:58:03.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:03.445: INFO: Found 0 / 1 Jan 8 06:58:04.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:04.445: INFO: Found 0 / 1 Jan 8 06:58:05.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:05.445: INFO: Found 0 / 1 Jan 8 06:58:06.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:06.445: INFO: Found 0 / 1 Jan 8 06:58:07.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:07.446: INFO: Found 0 / 1 Jan 8 06:58:08.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:08.445: INFO: Found 0 / 1 Jan 8 06:58:09.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:09.446: INFO: Found 0 / 1 Jan 8 06:58:10.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:10.444: INFO: Found 0 / 1 Jan 8 06:58:11.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:11.445: INFO: Found 0 / 1 Jan 8 06:58:12.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:12.445: INFO: Found 0 / 1 Jan 8 06:58:13.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:13.446: INFO: Found 0 / 1 Jan 8 06:58:14.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:14.445: INFO: Found 0 / 1 Jan 8 06:58:15.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:15.445: INFO: Found 0 / 1 Jan 8 06:58:16.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:16.445: INFO: Found 0 / 1 Jan 8 06:58:17.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:17.446: INFO: Found 0 / 1 Jan 8 06:58:18.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:18.445: INFO: Found 0 / 1 Jan 8 06:58:19.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:19.445: INFO: Found 0 / 1 Jan 8 06:58:20.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:20.445: INFO: Found 0 / 1 Jan 8 06:58:21.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:21.446: INFO: Found 0 / 1 Jan 8 06:58:22.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:22.444: INFO: Found 0 / 1 Jan 8 06:58:23.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:23.445: INFO: Found 0 / 1 Jan 8 06:58:24.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:24.445: INFO: Found 0 / 1 Jan 8 06:58:25.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:25.445: INFO: Found 0 / 1 Jan 8 06:58:26.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:26.445: INFO: Found 0 / 1 Jan 8 06:58:27.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:27.445: INFO: Found 0 / 1 Jan 8 06:58:28.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:28.445: INFO: Found 0 / 1 Jan 8 06:58:29.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:29.445: INFO: Found 0 / 1 Jan 8 06:58:30.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:30.446: INFO: Found 0 / 1 Jan 8 06:58:31.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:31.444: INFO: Found 0 / 1 Jan 8 06:58:32.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:32.444: INFO: Found 0 / 1 Jan 8 06:58:33.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:33.445: INFO: Found 0 / 1 Jan 8 06:58:34.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:34.445: INFO: Found 0 / 1 Jan 8 06:58:35.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:35.445: INFO: Found 0 / 1 Jan 8 06:58:36.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:36.445: INFO: Found 0 / 1 Jan 8 06:58:37.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:37.446: INFO: Found 0 / 1 Jan 8 06:58:38.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:38.446: INFO: Found 0 / 1 Jan 8 06:58:39.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:39.445: INFO: Found 0 / 1 Jan 8 06:58:40.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:40.445: INFO: Found 0 / 1 Jan 8 06:58:41.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:41.445: INFO: Found 0 / 1 Jan 8 06:58:42.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:42.445: INFO: Found 0 / 1 Jan 8 06:58:43.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:43.445: INFO: Found 0 / 1 Jan 8 06:58:44.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:44.445: INFO: Found 0 / 1 Jan 8 06:58:45.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:45.445: INFO: Found 0 / 1 Jan 8 06:58:46.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:46.445: INFO: Found 0 / 1 Jan 8 06:58:47.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:47.446: INFO: Found 0 / 1 Jan 8 06:58:48.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:48.445: INFO: Found 0 / 1 Jan 8 06:58:49.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:49.446: INFO: Found 0 / 1 Jan 8 06:58:50.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:50.445: INFO: Found 0 / 1 Jan 8 06:58:51.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:51.446: INFO: Found 0 / 1 Jan 8 06:58:52.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:52.445: INFO: Found 0 / 1 Jan 8 06:58:53.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:53.446: INFO: Found 0 / 1 Jan 8 06:58:54.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:54.445: INFO: Found 0 / 1 Jan 8 06:58:55.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:55.446: INFO: Found 0 / 1 Jan 8 06:58:56.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:56.445: INFO: Found 0 / 1 Jan 8 06:58:57.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:57.446: INFO: Found 0 / 1 Jan 8 06:58:58.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:58.446: INFO: Found 0 / 1 Jan 8 06:58:59.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:58:59.445: INFO: Found 0 / 1 Jan 8 06:59:00.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:00.445: INFO: Found 0 / 1 Jan 8 06:59:01.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:01.446: INFO: Found 0 / 1 Jan 8 06:59:02.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:02.446: INFO: Found 0 / 1 Jan 8 06:59:03.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:03.445: INFO: Found 0 / 1 Jan 8 06:59:04.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:04.445: INFO: Found 0 / 1 Jan 8 06:59:05.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:05.445: INFO: Found 0 / 1 Jan 8 06:59:06.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:06.445: INFO: Found 0 / 1 Jan 8 06:59:07.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:07.446: INFO: Found 0 / 1 Jan 8 06:59:08.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:08.445: INFO: Found 0 / 1 Jan 8 06:59:09.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:09.445: INFO: Found 0 / 1 Jan 8 06:59:10.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:10.445: INFO: Found 0 / 1 Jan 8 06:59:11.448: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:11.448: INFO: Found 0 / 1 Jan 8 06:59:12.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:12.446: INFO: Found 0 / 1 Jan 8 06:59:13.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:13.445: INFO: Found 0 / 1 Jan 8 06:59:14.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:14.445: INFO: Found 0 / 1 Jan 8 06:59:15.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:15.445: INFO: Found 0 / 1 Jan 8 06:59:16.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:16.445: INFO: Found 0 / 1 Jan 8 06:59:17.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:17.445: INFO: Found 0 / 1 Jan 8 06:59:18.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:18.444: INFO: Found 0 / 1 Jan 8 06:59:19.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:19.445: INFO: Found 0 / 1 Jan 8 06:59:20.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:20.446: INFO: Found 0 / 1 Jan 8 06:59:21.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:21.445: INFO: Found 0 / 1 Jan 8 06:59:22.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:22.445: INFO: Found 0 / 1 Jan 8 06:59:23.447: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:23.447: INFO: Found 0 / 1 Jan 8 06:59:24.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:24.445: INFO: Found 0 / 1 Jan 8 06:59:25.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:25.446: INFO: Found 0 / 1 Jan 8 06:59:26.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:26.445: INFO: Found 0 / 1 Jan 8 06:59:27.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:27.445: INFO: Found 0 / 1 Jan 8 06:59:28.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:28.445: INFO: Found 0 / 1 Jan 8 06:59:29.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:29.445: INFO: Found 0 / 1 Jan 8 06:59:30.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:30.445: INFO: Found 0 / 1 Jan 8 06:59:31.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:31.446: INFO: Found 0 / 1 Jan 8 06:59:32.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:32.445: INFO: Found 0 / 1 Jan 8 06:59:33.444: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:33.444: INFO: Found 0 / 1 Jan 8 06:59:34.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:34.445: INFO: Found 0 / 1 Jan 8 06:59:35.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:35.445: INFO: Found 0 / 1 Jan 8 06:59:36.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:36.445: INFO: Found 0 / 1 Jan 8 06:59:37.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:37.445: INFO: Found 0 / 1 Jan 8 06:59:38.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:38.445: INFO: Found 0 / 1 Jan 8 06:59:39.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:39.445: INFO: Found 0 / 1 Jan 8 06:59:40.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:40.445: INFO: Found 0 / 1 Jan 8 06:59:41.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:41.445: INFO: Found 0 / 1 Jan 8 06:59:42.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:42.445: INFO: Found 0 / 1 Jan 8 06:59:43.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:43.445: INFO: Found 0 / 1 Jan 8 06:59:44.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:44.445: INFO: Found 0 / 1 Jan 8 06:59:45.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:45.445: INFO: Found 0 / 1 Jan 8 06:59:46.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:46.446: INFO: Found 0 / 1 Jan 8 06:59:47.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:47.445: INFO: Found 0 / 1 Jan 8 06:59:48.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:48.446: INFO: Found 0 / 1 Jan 8 06:59:49.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:49.445: INFO: Found 0 / 1 Jan 8 06:59:50.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:50.445: INFO: Found 0 / 1 Jan 8 06:59:51.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:51.445: INFO: Found 0 / 1 Jan 8 06:59:52.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:52.445: INFO: Found 0 / 1 Jan 8 06:59:53.446: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:53.446: INFO: Found 0 / 1 Jan 8 06:59:54.445: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:54.445: INFO: Found 0 / 1 Jan 8 06:59:54.621: INFO: Selector matched 1 pods for map[app:agnhost] Jan 8 06:59:54.621: INFO: Found 0 / 1 Jan 8 06:59:54.621: INFO: WaitFor completed with timeout 5m0s. Pods found = 0 out of 1 �[1mSTEP:�[0m Collecting events from namespace "kubectl-4507". �[38;5;243m01/08/23 06:59:54.621�[0m �[1mSTEP:�[0m Found 12 events. �[38;5;243m01/08/23 06:59:54.798�[0m Jan 8 06:59:54.798: INFO: At 2023-01-08 06:54:54 +0000 UTC - event for agnhost-primary: {replication-controller } SuccessfulCreate: Created pod: agnhost-primary-9vrbd Jan 8 06:59:54.798: INFO: At 2023-01-08 06:54:54 +0000 UTC - event for agnhost-primary-9vrbd: {default-scheduler } Scheduled: Successfully assigned kubectl-4507/agnhost-primary-9vrbd to i-0685adc783a822be6 Jan 8 06:59:54.798: INFO: At 2023-01-08 06:54:54 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9ad23b7c026cd0610661599f3247df5fc5f83e79c8ba8639a06a9e59c5a5850f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:59:54.798: INFO: At 2023-01-08 06:55:07 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "629cbbcc87d9aa7395372c1da910eb7e63ce192b1b3a821ee16c610e435a69bf": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:59:54.798: INFO: At 2023-01-08 06:55:19 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d71e21aa5700f46656ac977452c25535dd7075c403014e83c4feddeb216be155": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:59:54.798: INFO: At 2023-01-08 06:55:31 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "775259d9d3fa770206a856ab5c191b09492fba0a2d4944a8917a374a7a16164b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:59:54.798: INFO: At 2023-01-08 06:55:42 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "497ffa2a9bb72f963cf2071fa085797fa78f57f328cff61d86583ea85f23fdd3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:59:54.798: INFO: At 2023-01-08 06:55:55 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "198e1ae92b61eeb159214dc4d939e1293022db81a73cc44a37ac90968eb6e8b6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:59:54.798: INFO: At 2023-01-08 06:56:09 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e53a2697512e3a56adb87e0a11c3010f0b3f8d5f1a5d50312226660d3b4a3871": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:59:54.798: INFO: At 2023-01-08 06:56:24 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9c90502e11c6a3754765ee7a03a5ff4a88914cbba8ba27b7b34f1beef2e4b52e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:59:54.798: INFO: At 2023-01-08 06:56:35 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a0225ab89bbb829e78753da0a3743b16aa8239312997e213bbc08132c2bdb89a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:59:54.798: INFO: At 2023-01-08 06:56:51 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "fa4fb21351ebb3d30730eee00f63d784e7808c29cab65c843a20a3d8f8d4d12a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:59:54.974: INFO: POD NODE PHASE GRACE CONDITIONS Jan 8 06:59:54.974: INFO: agnhost-primary-9vrbd i-0685adc783a822be6 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:54:54 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:54:54 +0000 UTC ContainersNotReady containers with unready status: [agnhost-primary]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:54:54 +0000 UTC ContainersNotReady containers with unready status: [agnhost-primary]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:54:54 +0000 UTC }] Jan 8 06:59:54.974: INFO: Jan 8 06:59:55.152: INFO: Unable to fetch kubectl-4507/agnhost-primary-9vrbd/agnhost-primary logs: the server rejected our request for an unknown reason (get pods agnhost-primary-9vrbd) Jan 8 06:59:55.329: INFO: Logging node info for node i-0685adc783a822be6 Jan 8 06:59:55.505: INFO: Node Info: &Node{ObjectMeta:{i-0685adc783a822be6 3eb588c7-cc76-44b4-b28a-492b9989254b 44173 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0685adc783a822be6 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0685adc783a822be6 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.112 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0685adc783a822be6"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:57:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-08 06:57:18 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0685adc783a822be6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:17 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:17 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:17 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:57:17 +0000 UTC,LastTransitionTime:2023-01-08 06:25:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.112,},NodeAddress{Type:ExternalIP,Address:43.201.46.149,},NodeAddress{Type:InternalDNS,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-201-46-149.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dccafbf198926b990228825ae0608,SystemUUID:ec2dccaf-bf19-8926-b990-228825ae0608,BootID:b8ff627c-af94-4730-be1d-6c7bfbfc818f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0488b0c43122e11c1 kubernetes.io/csi/ebs.csi.aws.com^vol-0ff9b2b6e0e0d5c29],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0488b0c43122e11c1,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0ff9b2b6e0e0d5c29,DevicePath:,},},Config:nil,},} Jan 8 06:59:55.506: INFO: Logging kubelet events for node i-0685adc783a822be6 Jan 8 06:59:55.684: INFO: Logging pods the kubelet thinks is on node i-0685adc783a822be6 Jan 8 06:59:56.045: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rrmzm started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:56.045: INFO: csi-mockplugin-0 started at 2023-01-08 06:56:47 +0000 UTC (0+3 container statuses recorded) Jan 8 06:59:56.045: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 06:59:56.045: INFO: Container driver-registrar ready: false, restart count 0 Jan 8 06:59:56.045: INFO: Container mock ready: false, restart count 0 Jan 8 06:59:56.045: INFO: pod-disruption-failure-ignore-0-w4rvn started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container c ready: false, restart count 0 Jan 8 06:59:56.045: INFO: pod-projected-secrets-907df958-d79f-42f9-87ee-41301a6f40be started at 2023-01-08 06:55:13 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container secret-volume-test ready: false, restart count 0 Jan 8 06:59:56.045: INFO: ebs-csi-node-ft7bw started at 2023-01-08 06:25:38 +0000 UTC (0+3 container statuses recorded) Jan 8 06:59:56.045: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:59:56.045: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:59:56.045: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:59:56.045: INFO: suspend-false-to-true-97wd6 started at 2023-01-08 06:53:18 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container c ready: true, restart count 0 Jan 8 06:59:56.045: INFO: csi-mockplugin-0 started at 2023-01-08 06:56:04 +0000 UTC (0+4 container statuses recorded) Jan 8 06:59:56.045: INFO: Container busybox ready: false, restart count 0 Jan 8 06:59:56.045: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 06:59:56.045: INFO: Container driver-registrar ready: false, restart count 0 Jan 8 06:59:56.045: INFO: Container mock ready: false, restart count 0 Jan 8 06:59:56.045: INFO: hostexec-i-0685adc783a822be6-ms669 started at 2023-01-08 06:56:52 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container agnhost-container ready: true, restart count 0 Jan 8 06:59:56.045: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-q97zd started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:56.045: INFO: pod-subpath-test-preprovisionedpv-xfcz started at 2023-01-08 06:58:38 +0000 UTC (2+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Init container init-volume-preprovisionedpv-xfcz ready: false, restart count 0 Jan 8 06:59:56.045: INFO: Init container test-init-subpath-preprovisionedpv-xfcz ready: false, restart count 0 Jan 8 06:59:56.045: INFO: Container test-container-subpath-preprovisionedpv-xfcz ready: false, restart count 0 Jan 8 06:59:56.045: INFO: service-proxy-disabled-6pp8x started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container service-proxy-disabled ready: true, restart count 0 Jan 8 06:59:56.045: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2bn4h started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:56.045: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v9tjx started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:56.045: INFO: affinity-clusterip-zxklh started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container affinity-clusterip ready: false, restart count 0 Jan 8 06:59:56.045: INFO: agnhost-primary-9vrbd started at 2023-01-08 06:54:54 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container agnhost-primary ready: false, restart count 0 Jan 8 06:59:56.045: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-97qp6 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:56.045: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-4d46c started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:56.045: INFO: csi-mockplugin-resizer-0 started at 2023-01-08 06:56:47 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container csi-resizer ready: false, restart count 0 Jan 8 06:59:56.045: INFO: pod-subpath-test-dynamicpv-28vt started at 2023-01-08 06:57:14 +0000 UTC (0+2 container statuses recorded) Jan 8 06:59:56.045: INFO: Container test-container-subpath-dynamicpv-28vt ready: false, restart count 0 Jan 8 06:59:56.045: INFO: Container test-container-volume-dynamicpv-28vt ready: false, restart count 0 Jan 8 06:59:56.045: INFO: hostexec-i-0685adc783a822be6-xj29s started at 2023-01-08 06:58:25 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container agnhost-container ready: true, restart count 0 Jan 8 06:59:56.045: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-zgmck started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:56.045: INFO: termination-message-container747db470-688a-41a3-b1bc-e438aac69648 started at 2023-01-08 06:58:30 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container termination-message-container ready: false, restart count 0 Jan 8 06:59:56.045: INFO: cilium-9vhbx started at 2023-01-08 06:25:38 +0000 UTC (1+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:59:56.045: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:59:56.045: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:56:47 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container csi-attacher ready: false, restart count 0 Jan 8 06:59:56.045: INFO: pod-disruption-failure-ignore-1-kfqqr started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container c ready: false, restart count 0 Jan 8 06:59:56.045: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-gq5pr started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:56.045: INFO: pod-8cb6de86-0572-49a0-a173-3eb48174b2d3 started at 2023-01-08 06:56:59 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container write-pod ready: false, restart count 0 Jan 8 06:59:56.045: INFO: inline-volume-tester-9qdkl started at 2023-01-08 06:52:11 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 8 06:59:56.045: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-svbp2 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:56.045: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mpw58 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:56.045: INFO: service-proxy-toggled-dh67n started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container service-proxy-toggled ready: false, restart count 0 Jan 8 06:59:56.045: INFO: liveness-51325787-aaa2-4ac1-a3f9-8ff47e3fa793 started at 2023-01-08 06:54:56 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:56.045: INFO: Container agnhost-container ready: false, restart count 0 Jan 8 06:59:56.698: INFO: Latency metrics for node i-0685adc783a822be6 Jan 8 06:59:56.698: INFO: Logging node info for node i-0bd196cab651da520 Jan 8 06:59:56.874: INFO: Node Info: &Node{ObjectMeta:{i-0bd196cab651da520 f7f00780-d24e-4c8e-b54d-373598e1d1d2 44470 0 2023-01-08 06:26:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0bd196cab651da520 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0bd196cab651da520 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.104 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bd196cab651da520"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:26:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:57:57 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-08 06:58:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0bd196cab651da520,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:58:03 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:58:03 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:58:03 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:58:03 +0000 UTC,LastTransitionTime:2023-01-08 06:26:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.104,},NodeAddress{Type:ExternalIP,Address:54.180.1.55,},NodeAddress{Type:InternalDNS,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-180-1-55.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec218f9d32a84b2feeb6301255fd576c,SystemUUID:ec218f9d-32a8-4b2f-eeb6-301255fd576c,BootID:accdf7fd-1195-4765-b28e-29d44bf47bac,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-04598dd4ddfa6789b kubernetes.io/csi/ebs.csi.aws.com^vol-06b18f9cd5a55aaf5 kubernetes.io/csi/ebs.csi.aws.com^vol-07a4e46ecc4ccd59e kubernetes.io/csi/ebs.csi.aws.com^vol-0d3ac7e97177e23e2],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d3ac7e97177e23e2,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-07a4e46ecc4ccd59e,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-04598dd4ddfa6789b,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-06b18f9cd5a55aaf5,DevicePath:,},},Config:nil,},} Jan 8 06:59:56.875: INFO: Logging kubelet events for node i-0bd196cab651da520 Jan 8 06:59:57.054: INFO: Logging pods the kubelet thinks is on node i-0bd196cab651da520 Jan 8 06:59:57.238: INFO: service-proxy-toggled-8nfkz started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container service-proxy-toggled ready: false, restart count 0 Jan 8 06:59:57.238: INFO: execpodgmcm4 started at 2023-01-08 06:55:35 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container agnhost-container ready: false, restart count 0 Jan 8 06:59:57.238: INFO: pod-cf6083c8-7a3d-4fd0-9a22-e056496319c3 started at 2023-01-08 06:57:54 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container write-pod ready: false, restart count 0 Jan 8 06:59:57.238: INFO: sample-webhook-deployment-865554f4d9-4kppl started at 2023-01-08 06:55:41 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container sample-webhook ready: false, restart count 0 Jan 8 06:59:57.238: INFO: csi-hostpathplugin-0 started at 2023-01-08 06:57:39 +0000 UTC (0+7 container statuses recorded) Jan 8 06:59:57.238: INFO: Container csi-attacher ready: false, restart count 0 Jan 8 06:59:57.238: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 06:59:57.238: INFO: Container csi-resizer ready: false, restart count 0 Jan 8 06:59:57.238: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 8 06:59:57.238: INFO: Container hostpath ready: false, restart count 0 Jan 8 06:59:57.238: INFO: Container liveness-probe ready: false, restart count 0 Jan 8 06:59:57.238: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 8 06:59:57.238: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-76nxw started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:57.238: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-75r8w started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:57.238: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v44bk started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:57.238: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9mp69 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:57.238: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-cddx5 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:57.238: INFO: suspend-false-to-true-xwpqd started at 2023-01-08 06:53:18 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container c ready: false, restart count 0 Jan 8 06:59:57.238: INFO: pod6b22b1b4-d027-4092-9a22-c1f06499be00 started at 2023-01-08 06:56:02 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container nginx ready: false, restart count 0 Jan 8 06:59:57.238: INFO: pod-subpath-test-dynamicpv-cx9n started at 2023-01-08 06:56:59 +0000 UTC (2+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Init container init-volume-dynamicpv-cx9n ready: false, restart count 0 Jan 8 06:59:57.238: INFO: Init container test-init-subpath-dynamicpv-cx9n ready: false, restart count 0 Jan 8 06:59:57.238: INFO: Container test-container-subpath-dynamicpv-cx9n ready: false, restart count 0 Jan 8 06:59:57.238: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-6df8l started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:57.238: INFO: inline-volume-tester-m9vk5 started at 2023-01-08 06:56:50 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 8 06:59:57.238: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-vrcjq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:57.238: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9776c started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:57.238: INFO: busybox-scheduling-0abca61c-e823-4136-8618-ce7deb358c0b started at 2023-01-08 06:55:28 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container busybox-scheduling-0abca61c-e823-4136-8618-ce7deb358c0b ready: false, restart count 0 Jan 8 06:59:57.238: INFO: ebs-csi-node-6grlm started at 2023-01-08 06:26:02 +0000 UTC (0+3 container statuses recorded) Jan 8 06:59:57.238: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:59:57.238: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:59:57.238: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:59:57.238: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9wskq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:57.238: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mphgx started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:57.238: INFO: pod1 started at 2023-01-08 06:51:10 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container agnhost-container ready: true, restart count 0 Jan 8 06:59:57.238: INFO: affinity-clusterip-mrptn started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Container affinity-clusterip ready: true, restart count 0 Jan 8 06:59:57.238: INFO: cilium-s6shx started at 2023-01-08 06:26:02 +0000 UTC (1+1 container statuses recorded) Jan 8 06:59:57.238: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:59:57.238: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:59:57.872: INFO: Latency metrics for node i-0bd196cab651da520 Jan 8 06:59:57.872: INFO: Logging node info for node i-0d149644e2edcf9cf Jan 8 06:59:58.048: INFO: Node Info: &Node{ObjectMeta:{i-0d149644e2edcf9cf b448ef72-6e61-4ad5-aa29-d63cc0f94945 43645 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0d149644e2edcf9cf kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0d149644e2edcf9cf topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.115 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0d149644e2edcf9cf"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:56:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0d149644e2edcf9cf,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.115,},NodeAddress{Type:ExternalIP,Address:3.35.219.218,},NodeAddress{Type:InternalDNS,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-35-219-218.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec26c6a9af66729e278545162595eb2d,SystemUUID:ec26c6a9-af66-729e-2785-45162595eb2d,BootID:959da9e3-fb86-442a-958f-2a49c509cdf4,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 8 06:59:58.049: INFO: Logging kubelet events for node i-0d149644e2edcf9cf Jan 8 06:59:58.227: INFO: Logging pods the kubelet thinks is on node i-0d149644e2edcf9cf Jan 8 06:59:58.410: INFO: ebs-csi-node-lfcwb started at 2023-01-08 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 8 06:59:58.410: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:59:58.410: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:59:58.410: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:59:58.410: INFO: coredns-559769c974-9q4qg started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:58.410: INFO: Container coredns ready: true, restart count 0 Jan 8 06:59:58.410: INFO: csi-hostpathplugin-0 started at 2023-01-08 06:52:05 +0000 UTC (0+7 container statuses recorded) Jan 8 06:59:58.410: INFO: Container csi-attacher ready: false, restart count 0 Jan 8 06:59:58.410: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 06:59:58.410: INFO: Container csi-resizer ready: false, restart count 0 Jan 8 06:59:58.410: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 8 06:59:58.410: INFO: Container hostpath ready: false, restart count 0 Jan 8 06:59:58.410: INFO: Container liveness-probe ready: false, restart count 0 Jan 8 06:59:58.410: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 8 06:59:58.410: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-xrmq4 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:58.410: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:58.410: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-7gwsq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:58.410: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:58.410: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nsfnm started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:58.410: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:58.410: INFO: coredns-autoscaler-7cb5c5b969-xqnk5 started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:58.410: INFO: Container autoscaler ready: true, restart count 0 Jan 8 06:59:58.410: INFO: service-proxy-disabled-v5f82 started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:58.410: INFO: Container service-proxy-disabled ready: true, restart count 0 Jan 8 06:59:58.410: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-tgn6r started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:58.410: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:58.410: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9b2xq started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:58.410: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:58.410: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-njh2f started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:58.410: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:58.410: INFO: cilium-58twl started at 2023-01-08 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 8 06:59:58.410: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:59:58.410: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:59:58.410: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rv5qq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:58.410: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:58.410: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-flhps started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:58.410: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:58.410: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2vh6v started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:58.410: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:59:58.410: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nt9jr started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:58.410: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0 Jan 8 06:59:59.120: INFO: Latency metrics for node i-0d149644e2edcf9cf Jan 8 06:59:59.120: INFO: Logging node info for node i-0eee6113fd9f81d94 Jan 8 06:59:59.297: INFO: Node Info: &Node{ObjectMeta:{i-0eee6113fd9f81d94 6de592fc-f11c-4ba3-afa0-707bb4ca29b2 44024 0 2023-01-08 06:23:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0eee6113fd9f81d94 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.48.64 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eee6113fd9f81d94"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-08 06:23:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-08 06:24:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-08 06:24:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-08 06:57:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0eee6113fd9f81d94,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:24:35 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.48.64,},NodeAddress{Type:ExternalIP,Address:3.35.206.95,},NodeAddress{Type:InternalDNS,Address:i-0eee6113fd9f81d94.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0eee6113fd9f81d94.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-35-206-95.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d8cf21861b15bb593210ff59f6044,SystemUUID:ec2d8cf2-1861-b15b-b593-210ff59f6044,BootID:fcad947c-a63f-4722-9a63-22eac74bea9f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43453935,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42803460,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967354,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 8 06:59:59.297: INFO: Logging kubelet events for node i-0eee6113fd9f81d94 Jan 8 06:59:59.476: INFO: Logging pods the kubelet thinks is on node i-0eee6113fd9f81d94 Jan 8 06:59:59.658: INFO: kops-controller-242qq started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:59.658: INFO: Container kops-controller ready: true, restart count 0 Jan 8 06:59:59.658: INFO: cilium-f22px started at 2023-01-08 06:24:21 +0000 UTC (1+1 container statuses recorded) Jan 8 06:59:59.658: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:59:59.658: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:59:59.658: INFO: etcd-manager-events-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:59.658: INFO: Container etcd-manager ready: true, restart count 0 Jan 8 06:59:59.658: INFO: etcd-manager-main-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:59.658: INFO: Container etcd-manager ready: true, restart count 0 Jan 8 06:59:59.658: INFO: kube-controller-manager-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:59.658: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 8 06:59:59.658: INFO: ebs-csi-node-nspqr started at 2023-01-08 06:24:21 +0000 UTC (0+3 container statuses recorded) Jan 8 06:59:59.658: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:59:59.658: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:59:59.658: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:59:59.658: INFO: aws-cloud-controller-manager-d8d56 started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:59.658: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 8 06:59:59.658: INFO: cilium-operator-5dd44dc49f-fd8vl started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:59.658: INFO: Container cilium-operator ready: true, restart count 0 Jan 8 06:59:59.658: INFO: ebs-csi-controller-89b649c94-vzzj9 started at 2023-01-08 06:24:21 +0000 UTC (0+5 container statuses recorded) Jan 8 06:59:59.658: INFO: Container csi-attacher ready: true, restart count 0 Jan 8 06:59:59.658: INFO: Container csi-provisioner ready: true, restart count 0 Jan 8 06:59:59.658: INFO: Container csi-resizer ready: true, restart count 0 Jan 8 06:59:59.658: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:59:59.658: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:59:59.658: INFO: dns-controller-69987775c6-v4cmg started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:59.658: INFO: Container dns-controller ready: true, restart count 0 Jan 8 06:59:59.658: INFO: kube-scheduler-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 06:59:59.658: INFO: Container kube-scheduler ready: true, restart count 0 Jan 8 06:59:59.658: INFO: kube-apiserver-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+2 container statuses recorded) Jan 8 06:59:59.658: INFO: Container healthcheck ready: true, restart count 0 Jan 8 06:59:59.658: INFO: Container kube-apiserver ready: true, restart count 1 Jan 8 07:00:00.262: INFO: Latency metrics for node i-0eee6113fd9f81d94 Jan 8 07:00:00.262: INFO: Logging node info for node i-0f469f08be563dad4 Jan 8 07:00:00.439: INFO: Node Info: &Node{ObjectMeta:{i-0f469f08be563dad4 fddb222c-a6ac-415f-83dc-4b71acb0e5ab 43138 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0f469f08be563dad4 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0f469f08be563dad4 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.57.173 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0f469f08be563dad4"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:55:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0f469f08be563dad4,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.57.173,},NodeAddress{Type:ExternalIP,Address:3.38.152.241,},NodeAddress{Type:InternalDNS,Address:i-0f469f08be563dad4.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f469f08be563dad4.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-38-152-241.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2abc7b1d816532cfb4787d85cb98ea,SystemUUID:ec2abc7b-1d81-6532-cfb4-787d85cb98ea,BootID:442daa6f-46d1-4c32-9182-8b6204c22aba,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 8 07:00:00.439: INFO: Logging kubelet events for node i-0f469f08be563dad4 Jan 8 07:00:00.618: INFO: Logging pods the kubelet thinks is on node i-0f469f08be563dad4 Jan 8 07:00:00.806: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-b6m8x started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:00.806: INFO: affinity-clusterip-8kclb started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Container affinity-clusterip ready: false, restart count 0 Jan 8 07:00:00.806: INFO: cilium-24rbm started at 2023-01-08 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 07:00:00.806: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 07:00:00.806: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-ktkbb started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:00.806: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-dvpgx started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:00.806: INFO: pod1 started at 2023-01-08 06:57:35 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Container agnhost-container ready: false, restart count 0 Jan 8 07:00:00.806: INFO: ebs-csi-node-9s8qc started at 2023-01-08 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 8 07:00:00.806: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 07:00:00.806: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 07:00:00.806: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 07:00:00.806: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-x88cb started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:00.806: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-h5c5c started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:00.806: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-jc58p started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:00.806: INFO: service-proxy-disabled-bqbxv started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Container service-proxy-disabled ready: true, restart count 0 Jan 8 07:00:00.806: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-gdlbp started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:00.806: INFO: service-proxy-toggled-7qcjj started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Container service-proxy-toggled ready: true, restart count 0 Jan 8 07:00:00.806: INFO: coredns-559769c974-f9s4x started at 2023-01-08 06:26:19 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Container coredns ready: true, restart count 0 Jan 8 07:00:00.806: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-glvzd started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:00.806: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-cwg4t started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:00.806: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-57rwf started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:00.806: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0 Jan 8 07:00:01.515: INFO: Latency metrics for node i-0f469f08be563dad4 Jan 8 07:00:01.515: FAIL: Verified 0 of 1 pods , error: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/kubectl.glob..func1.4(0x1) test/e2e/kubectl/kubectl.go:288 +0x13b k8s.io/kubernetes/test/e2e/kubectl.glob..func1.20.1() test/e2e/kubectl/kubectl.go:1427 +0x1b4 [AfterEach] [sig-cli] Kubectl client test/e2e/framework/node/init/init.go:32 Jan 8 07:00:01.516: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-cli] Kubectl client test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-cli] Kubectl client dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/08/23 07:00:01.693�[0m �[1mSTEP:�[0m Collecting events from namespace "kubectl-4507". �[38;5;243m01/08/23 07:00:01.693�[0m �[1mSTEP:�[0m Found 12 events. �[38;5;243m01/08/23 07:00:01.87�[0m Jan 8 07:00:01.870: INFO: At 2023-01-08 06:54:54 +0000 UTC - event for agnhost-primary: {replication-controller } SuccessfulCreate: Created pod: agnhost-primary-9vrbd Jan 8 07:00:01.870: INFO: At 2023-01-08 06:54:54 +0000 UTC - event for agnhost-primary-9vrbd: {default-scheduler } Scheduled: Successfully assigned kubectl-4507/agnhost-primary-9vrbd to i-0685adc783a822be6 Jan 8 07:00:01.870: INFO: At 2023-01-08 06:54:54 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9ad23b7c026cd0610661599f3247df5fc5f83e79c8ba8639a06a9e59c5a5850f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 07:00:01.870: INFO: At 2023-01-08 06:55:07 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "629cbbcc87d9aa7395372c1da910eb7e63ce192b1b3a821ee16c610e435a69bf": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 07:00:01.870: INFO: At 2023-01-08 06:55:19 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d71e21aa5700f46656ac977452c25535dd7075c403014e83c4feddeb216be155": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 07:00:01.870: INFO: At 2023-01-08 06:55:31 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "775259d9d3fa770206a856ab5c191b09492fba0a2d4944a8917a374a7a16164b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 07:00:01.870: INFO: At 2023-01-08 06:55:42 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "497ffa2a9bb72f963cf2071fa085797fa78f57f328cff61d86583ea85f23fdd3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 07:00:01.870: INFO: At 2023-01-08 06:55:55 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "198e1ae92b61eeb159214dc4d939e1293022db81a73cc44a37ac90968eb6e8b6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 07:00:01.870: INFO: At 2023-01-08 06:56:09 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e53a2697512e3a56adb87e0a11c3010f0b3f8d5f1a5d50312226660d3b4a3871": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 07:00:01.870: INFO: At 2023-01-08 06:56:24 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9c90502e11c6a3754765ee7a03a5ff4a88914cbba8ba27b7b34f1beef2e4b52e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 07:00:01.870: INFO: At 2023-01-08 06:56:35 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a0225ab89bbb829e78753da0a3743b16aa8239312997e213bbc08132c2bdb89a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 07:00:01.870: INFO: At 2023-01-08 06:56:51 +0000 UTC - event for agnhost-primary-9vrbd: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "fa4fb21351ebb3d30730eee00f63d784e7808c29cab65c843a20a3d8f8d4d12a": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 07:00:02.046: INFO: POD NODE PHASE GRACE CONDITIONS Jan 8 07:00:02.047: INFO: agnhost-primary-9vrbd i-0685adc783a822be6 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:54:54 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:54:54 +0000 UTC ContainersNotReady containers with unready status: [agnhost-primary]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:54:54 +0000 UTC ContainersNotReady containers with unready status: [agnhost-primary]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:54:54 +0000 UTC }] Jan 8 07:00:02.047: INFO: Jan 8 07:00:02.224: INFO: Unable to fetch kubectl-4507/agnhost-primary-9vrbd/agnhost-primary logs: the server rejected our request for an unknown reason (get pods agnhost-primary-9vrbd) Jan 8 07:00:02.403: INFO: Logging node info for node i-0685adc783a822be6 Jan 8 07:00:02.579: INFO: Node Info: &Node{ObjectMeta:{i-0685adc783a822be6 3eb588c7-cc76-44b4-b28a-492b9989254b 44173 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0685adc783a822be6 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0685adc783a822be6 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.112 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0685adc783a822be6"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:57:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-08 06:57:18 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0685adc783a822be6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:17 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:17 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:17 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:57:17 +0000 UTC,LastTransitionTime:2023-01-08 06:25:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.112,},NodeAddress{Type:ExternalIP,Address:43.201.46.149,},NodeAddress{Type:InternalDNS,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-201-46-149.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dccafbf198926b990228825ae0608,SystemUUID:ec2dccaf-bf19-8926-b990-228825ae0608,BootID:b8ff627c-af94-4730-be1d-6c7bfbfc818f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0488b0c43122e11c1 kubernetes.io/csi/ebs.csi.aws.com^vol-0ff9b2b6e0e0d5c29],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0488b0c43122e11c1,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0ff9b2b6e0e0d5c29,DevicePath:,},},Config:nil,},} Jan 8 07:00:02.580: INFO: Logging kubelet events for node i-0685adc783a822be6 Jan 8 07:00:02.760: INFO: Logging pods the kubelet thinks is on node i-0685adc783a822be6 Jan 8 07:00:02.955: INFO: cilium-9vhbx started at 2023-01-08 06:25:38 +0000 UTC (1+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 07:00:02.955: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 07:00:02.955: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:56:47 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container csi-attacher ready: false, restart count 0 Jan 8 07:00:02.955: INFO: pod-disruption-failure-ignore-1-kfqqr started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container c ready: false, restart count 0 Jan 8 07:00:02.955: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-gq5pr started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:02.955: INFO: pod-8cb6de86-0572-49a0-a173-3eb48174b2d3 started at 2023-01-08 06:56:59 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container write-pod ready: false, restart count 0 Jan 8 07:00:02.955: INFO: inline-volume-tester-9qdkl started at 2023-01-08 06:52:11 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 8 07:00:02.955: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-svbp2 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:02.955: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mpw58 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:02.955: INFO: service-proxy-toggled-dh67n started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container service-proxy-toggled ready: false, restart count 0 Jan 8 07:00:02.955: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rrmzm started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:02.955: INFO: csi-mockplugin-0 started at 2023-01-08 06:56:47 +0000 UTC (0+3 container statuses recorded) Jan 8 07:00:02.955: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 07:00:02.955: INFO: Container driver-registrar ready: false, restart count 0 Jan 8 07:00:02.955: INFO: Container mock ready: false, restart count 0 Jan 8 07:00:02.955: INFO: pod-disruption-failure-ignore-0-w4rvn started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container c ready: false, restart count 0 Jan 8 07:00:02.955: INFO: pod-projected-secrets-907df958-d79f-42f9-87ee-41301a6f40be started at 2023-01-08 06:55:13 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container secret-volume-test ready: false, restart count 0 Jan 8 07:00:02.955: INFO: ebs-csi-node-ft7bw started at 2023-01-08 06:25:38 +0000 UTC (0+3 container statuses recorded) Jan 8 07:00:02.955: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 07:00:02.955: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 07:00:02.955: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 07:00:02.955: INFO: suspend-false-to-true-97wd6 started at 2023-01-08 06:53:18 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container c ready: true, restart count 0 Jan 8 07:00:02.955: INFO: csi-mockplugin-0 started at 2023-01-08 06:56:04 +0000 UTC (0+4 container statuses recorded) Jan 8 07:00:02.955: INFO: Container busybox ready: false, restart count 0 Jan 8 07:00:02.955: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 07:00:02.955: INFO: Container driver-registrar ready: false, restart count 0 Jan 8 07:00:02.955: INFO: Container mock ready: false, restart count 0 Jan 8 07:00:02.955: INFO: hostexec-i-0685adc783a822be6-ms669 started at 2023-01-08 06:56:52 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container agnhost-container ready: true, restart count 0 Jan 8 07:00:02.955: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-q97zd started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:02.955: INFO: pod-subpath-test-preprovisionedpv-xfcz started at 2023-01-08 06:58:38 +0000 UTC (2+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Init container init-volume-preprovisionedpv-xfcz ready: false, restart count 0 Jan 8 07:00:02.955: INFO: Init container test-init-subpath-preprovisionedpv-xfcz ready: false, restart count 0 Jan 8 07:00:02.955: INFO: Container test-container-subpath-preprovisionedpv-xfcz ready: false, restart count 0 Jan 8 07:00:02.955: INFO: service-proxy-disabled-6pp8x started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container service-proxy-disabled ready: true, restart count 0 Jan 8 07:00:02.955: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2bn4h started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:02.955: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v9tjx started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:02.955: INFO: affinity-clusterip-zxklh started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container affinity-clusterip ready: false, restart count 0 Jan 8 07:00:02.955: INFO: agnhost-primary-9vrbd started at 2023-01-08 06:54:54 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container agnhost-primary ready: false, restart count 0 Jan 8 07:00:02.955: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-97qp6 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:02.955: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-4d46c started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:02.955: INFO: csi-mockplugin-resizer-0 started at 2023-01-08 06:56:47 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container csi-resizer ready: false, restart count 0 Jan 8 07:00:02.955: INFO: pod-subpath-test-dynamicpv-28vt started at 2023-01-08 06:57:14 +0000 UTC (0+2 container statuses recorded) Jan 8 07:00:02.955: INFO: Container test-container-subpath-dynamicpv-28vt ready: false, restart count 0 Jan 8 07:00:02.955: INFO: Container test-container-volume-dynamicpv-28vt ready: false, restart count 0 Jan 8 07:00:02.955: INFO: hostexec-i-0685adc783a822be6-xj29s started at 2023-01-08 06:58:25 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container agnhost-container ready: true, restart count 0 Jan 8 07:00:02.955: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-zgmck started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:02.955: INFO: termination-message-container747db470-688a-41a3-b1bc-e438aac69648 started at 2023-01-08 06:58:30 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:02.955: INFO: Container termination-message-container ready: false, restart count 0 Jan 8 07:00:03.777: INFO: Latency metrics for node i-0685adc783a822be6 Jan 8 07:00:03.777: INFO: Logging node info for node i-0bd196cab651da520 Jan 8 07:00:03.953: INFO: Node Info: &Node{ObjectMeta:{i-0bd196cab651da520 f7f00780-d24e-4c8e-b54d-373598e1d1d2 44470 0 2023-01-08 06:26:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0bd196cab651da520 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0bd196cab651da520 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.104 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bd196cab651da520"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:26:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:57:57 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-08 06:58:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0bd196cab651da520,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:58:03 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:58:03 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:58:03 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:58:03 +0000 UTC,LastTransitionTime:2023-01-08 06:26:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.104,},NodeAddress{Type:ExternalIP,Address:54.180.1.55,},NodeAddress{Type:InternalDNS,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-180-1-55.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec218f9d32a84b2feeb6301255fd576c,SystemUUID:ec218f9d-32a8-4b2f-eeb6-301255fd576c,BootID:accdf7fd-1195-4765-b28e-29d44bf47bac,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-04598dd4ddfa6789b kubernetes.io/csi/ebs.csi.aws.com^vol-06b18f9cd5a55aaf5 kubernetes.io/csi/ebs.csi.aws.com^vol-07a4e46ecc4ccd59e kubernetes.io/csi/ebs.csi.aws.com^vol-0d3ac7e97177e23e2],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0d3ac7e97177e23e2,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-07a4e46ecc4ccd59e,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-04598dd4ddfa6789b,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-06b18f9cd5a55aaf5,DevicePath:,},},Config:nil,},} Jan 8 07:00:03.954: INFO: Logging kubelet events for node i-0bd196cab651da520 Jan 8 07:00:04.133: INFO: Logging pods the kubelet thinks is on node i-0bd196cab651da520 Jan 8 07:00:04.317: INFO: pod-cf6083c8-7a3d-4fd0-9a22-e056496319c3 started at 2023-01-08 06:57:54 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container write-pod ready: false, restart count 0 Jan 8 07:00:04.317: INFO: service-proxy-toggled-8nfkz started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container service-proxy-toggled ready: false, restart count 0 Jan 8 07:00:04.317: INFO: execpodgmcm4 started at 2023-01-08 06:55:35 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container agnhost-container ready: false, restart count 0 Jan 8 07:00:04.317: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-76nxw started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:04.317: INFO: sample-webhook-deployment-865554f4d9-4kppl started at 2023-01-08 06:55:41 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container sample-webhook ready: false, restart count 0 Jan 8 07:00:04.317: INFO: csi-hostpathplugin-0 started at 2023-01-08 06:57:39 +0000 UTC (0+7 container statuses recorded) Jan 8 07:00:04.317: INFO: Container csi-attacher ready: false, restart count 0 Jan 8 07:00:04.317: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 07:00:04.317: INFO: Container csi-resizer ready: false, restart count 0 Jan 8 07:00:04.317: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 8 07:00:04.317: INFO: Container hostpath ready: false, restart count 0 Jan 8 07:00:04.317: INFO: Container liveness-probe ready: false, restart count 0 Jan 8 07:00:04.317: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 8 07:00:04.317: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v44bk started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:04.317: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-75r8w started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:04.317: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9mp69 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:04.317: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-cddx5 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:04.317: INFO: suspend-false-to-true-xwpqd started at 2023-01-08 06:53:18 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container c ready: false, restart count 0 Jan 8 07:00:04.317: INFO: pod6b22b1b4-d027-4092-9a22-c1f06499be00 started at 2023-01-08 06:56:02 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container nginx ready: false, restart count 0 Jan 8 07:00:04.317: INFO: pod-subpath-test-dynamicpv-cx9n started at 2023-01-08 06:56:59 +0000 UTC (2+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Init container init-volume-dynamicpv-cx9n ready: false, restart count 0 Jan 8 07:00:04.317: INFO: Init container test-init-subpath-dynamicpv-cx9n ready: false, restart count 0 Jan 8 07:00:04.317: INFO: Container test-container-subpath-dynamicpv-cx9n ready: false, restart count 0 Jan 8 07:00:04.317: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-6df8l started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:04.317: INFO: busybox-scheduling-0abca61c-e823-4136-8618-ce7deb358c0b started at 2023-01-08 06:55:28 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container busybox-scheduling-0abca61c-e823-4136-8618-ce7deb358c0b ready: false, restart count 0 Jan 8 07:00:04.317: INFO: inline-volume-tester-m9vk5 started at 2023-01-08 06:56:50 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 8 07:00:04.317: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-vrcjq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:04.317: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9776c started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:04.317: INFO: affinity-clusterip-mrptn started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container affinity-clusterip ready: true, restart count 0 Jan 8 07:00:04.317: INFO: cilium-s6shx started at 2023-01-08 06:26:02 +0000 UTC (1+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 07:00:04.317: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 07:00:04.317: INFO: ebs-csi-node-6grlm started at 2023-01-08 06:26:02 +0000 UTC (0+3 container statuses recorded) Jan 8 07:00:04.317: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 07:00:04.317: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 07:00:04.317: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 07:00:04.317: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9wskq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:04.317: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mphgx started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:04.317: INFO: pod1 started at 2023-01-08 06:51:10 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:04.317: INFO: Container agnhost-container ready: true, restart count 0 Jan 8 07:00:05.085: INFO: Latency metrics for node i-0bd196cab651da520 Jan 8 07:00:05.085: INFO: Logging node info for node i-0d149644e2edcf9cf Jan 8 07:00:05.265: INFO: Node Info: &Node{ObjectMeta:{i-0d149644e2edcf9cf b448ef72-6e61-4ad5-aa29-d63cc0f94945 43645 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0d149644e2edcf9cf kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0d149644e2edcf9cf topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.115 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0d149644e2edcf9cf"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:56:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0d149644e2edcf9cf,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:56:25 +0000 UTC,LastTransitionTime:2023-01-08 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.115,},NodeAddress{Type:ExternalIP,Address:3.35.219.218,},NodeAddress{Type:InternalDNS,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-35-219-218.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec26c6a9af66729e278545162595eb2d,SystemUUID:ec26c6a9-af66-729e-2785-45162595eb2d,BootID:959da9e3-fb86-442a-958f-2a49c509cdf4,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 8 07:00:05.265: INFO: Logging kubelet events for node i-0d149644e2edcf9cf Jan 8 07:00:05.444: INFO: Logging pods the kubelet thinks is on node i-0d149644e2edcf9cf Jan 8 07:00:05.626: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-njh2f started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:05.626: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:05.626: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rv5qq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:05.626: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:05.626: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-flhps started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:05.626: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:05.626: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2vh6v started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:05.626: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:05.626: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nt9jr started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:05.626: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0 Jan 8 07:00:05.626: INFO: cilium-58twl started at 2023-01-08 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 8 07:00:05.626: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 07:00:05.626: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 07:00:05.626: INFO: coredns-559769c974-9q4qg started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:05.626: INFO: Container coredns ready: true, restart count 0 Jan 8 07:00:05.626: INFO: csi-hostpathplugin-0 started at 2023-01-08 06:52:05 +0000 UTC (0+7 container statuses recorded) Jan 8 07:00:05.626: INFO: Container csi-attacher ready: false, restart count 0 Jan 8 07:00:05.626: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 07:00:05.626: INFO: Container csi-resizer ready: false, restart count 0 Jan 8 07:00:05.626: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 8 07:00:05.626: INFO: Container hostpath ready: false, restart count 0 Jan 8 07:00:05.626: INFO: Container liveness-probe ready: false, restart count 0 Jan 8 07:00:05.626: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 8 07:00:05.626: INFO: ebs-csi-node-lfcwb started at 2023-01-08 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 8 07:00:05.626: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 07:00:05.626: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 07:00:05.626: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 07:00:05.626: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-7gwsq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:05.626: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:05.626: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nsfnm started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:05.626: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:05.626: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-xrmq4 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:05.626: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:05.626: INFO: service-proxy-disabled-v5f82 started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:05.626: INFO: Container service-proxy-disabled ready: true, restart count 0 Jan 8 07:00:05.626: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-tgn6r started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:05.626: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:05.626: INFO: coredns-autoscaler-7cb5c5b969-xqnk5 started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:05.626: INFO: Container autoscaler ready: true, restart count 0 Jan 8 07:00:05.626: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9b2xq started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:05.626: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:06.433: INFO: Latency metrics for node i-0d149644e2edcf9cf Jan 8 07:00:06.433: INFO: Logging node info for node i-0eee6113fd9f81d94 Jan 8 07:00:06.609: INFO: Node Info: &Node{ObjectMeta:{i-0eee6113fd9f81d94 6de592fc-f11c-4ba3-afa0-707bb4ca29b2 44024 0 2023-01-08 06:23:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0eee6113fd9f81d94 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.48.64 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eee6113fd9f81d94"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-08 06:23:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-08 06:24:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-08 06:24:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-08 06:57:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0eee6113fd9f81d94,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:57:02 +0000 UTC,LastTransitionTime:2023-01-08 06:24:35 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.48.64,},NodeAddress{Type:ExternalIP,Address:3.35.206.95,},NodeAddress{Type:InternalDNS,Address:i-0eee6113fd9f81d94.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0eee6113fd9f81d94.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-35-206-95.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d8cf21861b15bb593210ff59f6044,SystemUUID:ec2d8cf2-1861-b15b-b593-210ff59f6044,BootID:fcad947c-a63f-4722-9a63-22eac74bea9f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43453935,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42803460,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967354,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 8 07:00:06.610: INFO: Logging kubelet events for node i-0eee6113fd9f81d94 Jan 8 07:00:06.788: INFO: Logging pods the kubelet thinks is on node i-0eee6113fd9f81d94 Jan 8 07:00:06.972: INFO: etcd-manager-events-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:06.972: INFO: Container etcd-manager ready: true, restart count 0 Jan 8 07:00:06.972: INFO: etcd-manager-main-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:06.972: INFO: Container etcd-manager ready: true, restart count 0 Jan 8 07:00:06.972: INFO: kops-controller-242qq started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:06.972: INFO: Container kops-controller ready: true, restart count 0 Jan 8 07:00:06.972: INFO: cilium-f22px started at 2023-01-08 06:24:21 +0000 UTC (1+1 container statuses recorded) Jan 8 07:00:06.972: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 07:00:06.972: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 07:00:06.972: INFO: cilium-operator-5dd44dc49f-fd8vl started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:06.972: INFO: Container cilium-operator ready: true, restart count 0 Jan 8 07:00:06.972: INFO: ebs-csi-controller-89b649c94-vzzj9 started at 2023-01-08 06:24:21 +0000 UTC (0+5 container statuses recorded) Jan 8 07:00:06.972: INFO: Container csi-attacher ready: true, restart count 0 Jan 8 07:00:06.972: INFO: Container csi-provisioner ready: true, restart count 0 Jan 8 07:00:06.972: INFO: Container csi-resizer ready: true, restart count 0 Jan 8 07:00:06.972: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 07:00:06.972: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 07:00:06.972: INFO: dns-controller-69987775c6-v4cmg started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:06.972: INFO: Container dns-controller ready: true, restart count 0 Jan 8 07:00:06.972: INFO: kube-scheduler-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:06.972: INFO: Container kube-scheduler ready: true, restart count 0 Jan 8 07:00:06.972: INFO: kube-apiserver-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+2 container statuses recorded) Jan 8 07:00:06.972: INFO: Container healthcheck ready: true, restart count 0 Jan 8 07:00:06.972: INFO: Container kube-apiserver ready: true, restart count 1 Jan 8 07:00:06.972: INFO: kube-controller-manager-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:06.972: INFO: Container kube-controller-manager ready: true, restart count 2 Jan 8 07:00:06.972: INFO: ebs-csi-node-nspqr started at 2023-01-08 06:24:21 +0000 UTC (0+3 container statuses recorded) Jan 8 07:00:06.972: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 07:00:06.972: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 07:00:06.972: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 07:00:06.972: INFO: aws-cloud-controller-manager-d8d56 started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:06.972: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 8 07:00:07.569: INFO: Latency metrics for node i-0eee6113fd9f81d94 Jan 8 07:00:07.569: INFO: Logging node info for node i-0f469f08be563dad4 Jan 8 07:00:07.745: INFO: Node Info: &Node{ObjectMeta:{i-0f469f08be563dad4 fddb222c-a6ac-415f-83dc-4b71acb0e5ab 45013 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0f469f08be563dad4 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0f469f08be563dad4 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.57.173 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0f469f08be563dad4"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2023-01-08 06:55:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0f469f08be563dad4,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:55:06 +0000 UTC,LastTransitionTime:2023-01-08 06:25:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.57.173,},NodeAddress{Type:ExternalIP,Address:3.38.152.241,},NodeAddress{Type:InternalDNS,Address:i-0f469f08be563dad4.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f469f08be563dad4.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-38-152-241.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2abc7b1d816532cfb4787d85cb98ea,SystemUUID:ec2abc7b-1d81-6532-cfb4-787d85cb98ea,BootID:442daa6f-46d1-4c32-9182-8b6204c22aba,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 8 07:00:07.746: INFO: Logging kubelet events for node i-0f469f08be563dad4 Jan 8 07:00:07.924: INFO: Logging pods the kubelet thinks is on node i-0f469f08be563dad4 Jan 8 07:00:08.106: INFO: service-proxy-disabled-bqbxv started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Container service-proxy-disabled ready: true, restart count 0 Jan 8 07:00:08.106: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-gdlbp started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:08.106: INFO: service-proxy-toggled-7qcjj started at 2023-01-08 06:52:31 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Container service-proxy-toggled ready: true, restart count 0 Jan 8 07:00:08.106: INFO: coredns-559769c974-f9s4x started at 2023-01-08 06:26:19 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Container coredns ready: true, restart count 0 Jan 8 07:00:08.106: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-glvzd started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:08.106: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-cwg4t started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:08.106: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-57rwf started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0 Jan 8 07:00:08.106: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-b6m8x started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:08.106: INFO: affinity-clusterip-8kclb started at 2023-01-08 06:52:25 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Container affinity-clusterip ready: false, restart count 0 Jan 8 07:00:08.106: INFO: cilium-24rbm started at 2023-01-08 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 07:00:08.106: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 07:00:08.106: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-ktkbb started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:08.106: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-dvpgx started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:08.106: INFO: pod1 started at 2023-01-08 06:57:35 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Container agnhost-container ready: false, restart count 0 Jan 8 07:00:08.106: INFO: ebs-csi-node-9s8qc started at 2023-01-08 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 8 07:00:08.106: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 07:00:08.106: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 07:00:08.106: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 07:00:08.106: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-x88cb started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:08.106: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-h5c5c started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:08.106: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-jc58p started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 07:00:08.106: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 07:00:08.743: INFO: Latency metrics for node i-0f469f08be563dad4 [DeferCleanup (Each)] [sig-cli] Kubectl client tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "kubectl-4507" for this suite. �[38;5;243m01/08/23 07:00:08.743�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sNetworking\sGranular\sChecks\:\sServices\sshould\sbe\sable\sto\shandle\slarge\srequests\:\sudp$'
test/e2e/framework/network/utils.go:866
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc0001f6460, {0x75c6f5c, 0x9}, 0xc003017710)
test/e2e/framework/network/utils.go:866 +0x1d0
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc0001f6460, 0x7ffa2c761738?)
test/e2e/framework/network/utils.go:763 +0x55
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc0001f6460, 0x3e?)
test/e2e/framework/network/utils.go:778 +0x3e
k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc000f5e690, {0x0, 0x0, 0x0?})
test/e2e/framework/network/utils.go:131 +0x125
k8s.io/kubernetes/test/e2e/network.glob..func22.6.19()
test/e2e/network/networking.go:462 +0x36
[BeforeEach] [sig-network] Networking
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/08/23 06:46:32.215�[0m
Jan 8 06:46:32.215: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename nettest �[38;5;243m01/08/23 06:46:32.216�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/08/23 06:46:32.75�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/08/23 06:46:33.104�[0m
[BeforeEach] [sig-network] Networking
test/e2e/framework/metrics/init/init.go:31
[It] should be able to handle large requests: udp
test/e2e/network/networking.go:461
�[1mSTEP:�[0m Performing setup for networking test in namespace nettest-7775 �[38;5;243m01/08/23 06:46:33.459�[0m
�[1mSTEP:�[0m creating a selector �[38;5;243m01/08/23 06:46:33.459�[0m
�[1mSTEP:�[0m Creating the service pods in kubernetes �[38;5;243m01/08/23 06:46:33.459�[0m
Jan 8 06:46:33.459: INFO: Waiting up to 10m0s for all (but 0) nodes to be schedulable
Jan 8 06:46:34.540: INFO: Waiting up to 5m0s for pod "netserver-0" in namespace "nettest-7775" to be "running and ready"
Jan 8 06:46:34.718: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 177.969516ms
Jan 8 06:46:34.718: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:46:36.899: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2.359234181s
Jan 8 06:46:36.899: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:46:38.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4.356187094s
Jan 8 06:46:38.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:46:40.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 6.356713812s
Jan 8 06:46:40.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:46:42.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 8.356932074s
Jan 8 06:46:42.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:46:44.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 10.356267179s
Jan 8 06:46:44.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:46:46.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 12.356037566s
Jan 8 06:46:46.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:46:48.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 14.355712037s
Jan 8 06:46:48.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:46:50.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 16.356016422s
Jan 8 06:46:50.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:46:52.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 18.356256948s
Jan 8 06:46:52.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:46:54.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 20.356517555s
Jan 8 06:46:54.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:46:56.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 22.357060008s
Jan 8 06:46:56.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:46:58.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 24.356404042s
Jan 8 06:46:58.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:00.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 26.356198744s
Jan 8 06:47:00.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:02.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 28.356554893s
Jan 8 06:47:02.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:04.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 30.356555131s
Jan 8 06:47:04.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:06.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 32.356323731s
Jan 8 06:47:06.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:08.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 34.356262829s
Jan 8 06:47:08.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:10.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 36.356813034s
Jan 8 06:47:10.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:12.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 38.356274147s
Jan 8 06:47:12.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:14.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 40.356428892s
Jan 8 06:47:14.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:16.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 42.356223218s
Jan 8 06:47:16.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:18.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 44.356593139s
Jan 8 06:47:18.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:20.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 46.356932091s
Jan 8 06:47:20.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:22.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 48.355767339s
Jan 8 06:47:22.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:24.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 50.356333708s
Jan 8 06:47:24.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:26.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 52.355909903s
Jan 8 06:47:26.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:28.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 54.356182108s
Jan 8 06:47:28.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:30.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 56.356716296s
Jan 8 06:47:30.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:32.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 58.356057611s
Jan 8 06:47:32.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:34.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.356843555s
Jan 8 06:47:34.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:36.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.357117197s
Jan 8 06:47:36.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:38.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.356424505s
Jan 8 06:47:38.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:40.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.356529876s
Jan 8 06:47:40.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:42.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.355829298s
Jan 8 06:47:42.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:44.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.356005671s
Jan 8 06:47:44.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:46.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.356912916s
Jan 8 06:47:46.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:48.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.355788633s
Jan 8 06:47:48.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:50.900: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.359703747s
Jan 8 06:47:50.900: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:52.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.356085507s
Jan 8 06:47:52.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:54.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.356379409s
Jan 8 06:47:54.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:56.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.356113395s
Jan 8 06:47:56.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:47:58.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.356307646s
Jan 8 06:47:58.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:00.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.356343062s
Jan 8 06:48:00.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:02.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.356519879s
Jan 8 06:48:02.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:04.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.356062921s
Jan 8 06:48:04.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:06.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.357253197s
Jan 8 06:48:06.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:08.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.356626125s
Jan 8 06:48:08.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:10.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.356371079s
Jan 8 06:48:10.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:12.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.356961822s
Jan 8 06:48:12.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:14.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.356396237s
Jan 8 06:48:14.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:16.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.356481995s
Jan 8 06:48:16.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:18.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.356027244s
Jan 8 06:48:18.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:20.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.356263708s
Jan 8 06:48:20.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:22.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.356739226s
Jan 8 06:48:22.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:24.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.356441564s
Jan 8 06:48:24.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:26.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.356998468s
Jan 8 06:48:26.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:28.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.357030724s
Jan 8 06:48:28.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:30.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.355926649s
Jan 8 06:48:30.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:32.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.356456193s
Jan 8 06:48:32.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:34.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.356776541s
Jan 8 06:48:34.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:36.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.356691177s
Jan 8 06:48:36.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:38.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.356194835s
Jan 8 06:48:38.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:40.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.355993489s
Jan 8 06:48:40.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:42.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.356125828s
Jan 8 06:48:42.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:44.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.356356132s
Jan 8 06:48:44.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:46.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.356117606s
Jan 8 06:48:46.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:48.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.355974455s
Jan 8 06:48:48.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:50.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.356442854s
Jan 8 06:48:50.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:52.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.356661307s
Jan 8 06:48:52.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:54.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.357617599s
Jan 8 06:48:54.898: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:56.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.357376786s
Jan 8 06:48:56.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:48:58.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.356789243s
Jan 8 06:48:58.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:00.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.356660403s
Jan 8 06:49:00.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:02.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.357134571s
Jan 8 06:49:02.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:04.898: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.358027549s
Jan 8 06:49:04.898: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:06.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.356547588s
Jan 8 06:49:06.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:08.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.355863548s
Jan 8 06:49:08.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:10.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.356190061s
Jan 8 06:49:10.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:12.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.355876877s
Jan 8 06:49:12.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:14.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.356356593s
Jan 8 06:49:14.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:16.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.356855532s
Jan 8 06:49:16.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:18.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.355832232s
Jan 8 06:49:18.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:20.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.356113004s
Jan 8 06:49:20.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:22.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.35676168s
Jan 8 06:49:22.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:24.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.356054635s
Jan 8 06:49:24.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:26.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.356029758s
Jan 8 06:49:26.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:28.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.356282647s
Jan 8 06:49:28.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:30.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.355936258s
Jan 8 06:49:30.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:32.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.356390052s
Jan 8 06:49:32.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:34.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.356110481s
Jan 8 06:49:34.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:36.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.357114604s
Jan 8 06:49:36.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:38.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.356337045s
Jan 8 06:49:38.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:40.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.356204899s
Jan 8 06:49:40.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:42.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.356127259s
Jan 8 06:49:42.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:44.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.356733921s
Jan 8 06:49:44.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:46.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.35689187s
Jan 8 06:49:46.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:48.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.355715502s
Jan 8 06:49:48.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:50.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.356007071s
Jan 8 06:49:50.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:52.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.356138748s
Jan 8 06:49:52.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:54.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.356626807s
Jan 8 06:49:54.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:56.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.356313707s
Jan 8 06:49:56.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:49:58.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.355915355s
Jan 8 06:49:58.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:00.898: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.357938678s
Jan 8 06:50:00.898: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:02.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.356760712s
Jan 8 06:50:02.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:04.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.356076037s
Jan 8 06:50:04.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:06.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.356324128s
Jan 8 06:50:06.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:08.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.357195381s
Jan 8 06:50:08.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:10.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.356748376s
Jan 8 06:50:10.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:12.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.355857827s
Jan 8 06:50:12.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:14.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.356438519s
Jan 8 06:50:14.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:16.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.355694646s
Jan 8 06:50:16.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:18.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.356809428s
Jan 8 06:50:18.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:20.898: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.357630558s
Jan 8 06:50:20.898: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:22.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.355819366s
Jan 8 06:50:22.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:24.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.356265346s
Jan 8 06:50:24.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:26.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.355847076s
Jan 8 06:50:26.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:28.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.355888597s
Jan 8 06:50:28.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:30.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.356865284s
Jan 8 06:50:30.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:32.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.355992316s
Jan 8 06:50:32.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:34.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.35593379s
Jan 8 06:50:34.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:36.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.356768043s
Jan 8 06:50:36.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:38.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.356222837s
Jan 8 06:50:38.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:40.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.355683247s
Jan 8 06:50:40.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:42.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.356377908s
Jan 8 06:50:42.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:44.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.355944764s
Jan 8 06:50:44.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:46.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.356467243s
Jan 8 06:50:46.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:48.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.356783643s
Jan 8 06:50:48.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:50.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.355840828s
Jan 8 06:50:50.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:52.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.357084898s
Jan 8 06:50:52.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:54.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.355903384s
Jan 8 06:50:54.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:56.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.356017959s
Jan 8 06:50:56.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:50:58.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.356366408s
Jan 8 06:50:58.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:00.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.355912997s
Jan 8 06:51:00.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:02.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.356141692s
Jan 8 06:51:02.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:04.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.356755966s
Jan 8 06:51:04.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:06.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.357023881s
Jan 8 06:51:06.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:08.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.356242971s
Jan 8 06:51:08.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:10.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.356094954s
Jan 8 06:51:10.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:12.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.356243011s
Jan 8 06:51:12.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:14.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.35625864s
Jan 8 06:51:14.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:16.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.356613611s
Jan 8 06:51:16.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:18.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.356800651s
Jan 8 06:51:18.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:20.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.356888214s
Jan 8 06:51:20.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:22.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.356133999s
Jan 8 06:51:22.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:24.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.35691594s
Jan 8 06:51:24.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:26.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.356290173s
Jan 8 06:51:26.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:28.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.356164967s
Jan 8 06:51:28.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:30.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.356403347s
Jan 8 06:51:30.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:32.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.356556121s
Jan 8 06:51:32.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:34.896: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.355638242s
Jan 8 06:51:34.896: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:35.073: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.533256071s
Jan 8 06:51:35.073: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 8 06:51:35.074: INFO: Unexpected error:
<*pod.timeoutError | 0xc003016450>: {
msg: "timed out while waiting for pod nettest-7775/netserver-0 to be running and ready",
observedObjects: [
<*v1.Pod | 0xc001a9b680>{
TypeMeta: {Kind: "", APIVersion: ""},
ObjectMeta: {
Name: "netserver-0",
GenerateName: "",
Namespace: "nettest-7775",
SelfLink: "",
UID: "c80cf923-5ce7-4c86-a66e-c84846c90fb1",
ResourceVersion: "39794",
Generation: 0,
CreationTimestamp: {
Time: {
wall: 0,
ext: 63808757193,
loc: {
name: "Local",
zone: [
{name: "UTC", offset: 0, isDST: false},
],
tx: [
{
when: -576460752303423488,
index: 0,
isstd: false,
isutc: false,
},
],
extend: "UTC0",
cacheStart: 9223372036854775807,
cacheEnd: 9223372036854775807,
cacheZone: {name: "UTC", offset: 0, isDST: false},
},
},
},
DeletionTimestamp: nil,
DeletionGracePeriodSeconds: nil,
Labels: {
"selector-2a3e2fe8-8ff8-4765-bac5-8dd025fff41c": "true",
},
Annotations: nil,
OwnerReferences: nil,
Finalizers: nil,
ManagedFields: [
{
Manager: "e2e.test",
Operation: "Update",
APIVersion: "v1",
Time: {
Time: {
wall: 0,
ext: 63808757193,
loc: {
name: "Local",
zone: [...],
tx: [...],
extend: "UTC0",
cacheStart: 9223372036854775807,
cacheEnd: 9223372036854775807,
cacheZone: {name: ..., offset: ..., isDST: ...},
},
},
},
FieldsType: "FieldsV1",
FieldsV1: {
Raw: "{\"f:metadata\":{\"f:labels\":{\".\":{},\"f:selector-2a3e2fe8-8ff8-4765-bac5-8dd025fff41c\":{}}},\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"webserver\\\"}\":{\".\":{},\"f:args\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:livenessProbe\":{\".\":{},\"f:failureThreshold\":{},\"f:httpGet\":{\".\":{},\"f:path\":{},\"f:port\":{},\"f:scheme\":{}},\"f:initialDelaySeconds\":{},\"f:periodSeconds\":{},\"f:successThreshold\":{},\"f:timeoutSeconds\":{}},\"f:name\":{},\"f:ports\":{\".\":{},\"k:{\\\"containerPort\\\":8081,\\\"protocol\\\":\\\"UDP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:name\":{},\"f:protocol\":{}},\"k:{\\\"containerPort\\\":8083,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:name\":{},\"f:protocol\":{}}},\"f:readinessProbe\":{\".\":{},\"f:failureThreshold\":{},\"f:httpGet\":{\".\":{},\"f:path\":{},\"f:port\":{},\"f:scheme\":{}},\"f:initialDelaySeconds\":{},\"f:periodSeconds\":{},\"f:successThreshold\":{},\"f:timeoutSeconds\":{}},\"f:resources\":{},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:nodeSelector\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:terminationGracePeriodSeconds\":{}}}",
},
Subresource: "",
},
{
Manager: "kubelet",
Operation: "Update",
APIVersion: "v1",
Time: {
Time: {
wall: 0,
ext: 63808757193,
loc: {
name: "Local",
zone: [...],
tx: [...],
extend: "UTC0",
cacheStart: 9223372036854775807,
cacheEnd: 9223372036854775807,
cacheZone: {name: ..., offset: ..., isDST: ...},
},
},
},
FieldsType: "FieldsV1",
FieldsV1: {
Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}",
},
Subresource: "status",
},
],
},
Spec: {
Volumes: [
{
Name: "kube-api-access-ncn8g",
VolumeSource: {
HostPath: nil,
EmptyDir: nil,
GCEPersistentDisk: nil,
AWSElasticBlockStore: nil,
GitRepo: nil,
Secret: nil,
NFS: nil,
ISCSI: nil,
Glusterfs: nil,
PersistentVolumeClaim: nil,
RBD: nil,
FlexVolume: nil,
Cinder: nil,
CephFS: nil,
Flocker: nil,
DownwardAPI: nil,
FC: nil,
AzureFile: nil,
ConfigMap: nil,
VsphereVolume: nil,
Quobyte: nil,
AzureDisk: nil,
PhotonPersistentDisk: nil,
Projected: {
Sources: [
{
Secret: ...,
DownwardAPI: ...,
ConfigMap: ...,
ServiceAccountToken: ...,
},
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
Jan 8 06:51:35.075: FAIL: timed out while waiting for pod nettest-7775/netserver-0 to be running and ready
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc0001f6460, {0x75c6f5c, 0x9}, 0xc003017710)
test/e2e/framework/network/utils.go:866 +0x1d0
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc0001f6460, 0x7ffa2c761738?)
test/e2e/framework/network/utils.go:763 +0x55
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc0001f6460, 0x3e?)
test/e2e/framework/network/utils.go:778 +0x3e
k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc000f5e690, {0x0, 0x0, 0x0?})
test/e2e/framework/network/utils.go:131 +0x125
k8s.io/kubernetes/test/e2e/network.glob..func22.6.19()
test/e2e/network/networking.go:462 +0x36
[AfterEach] [sig-network] Networking
test/e2e/framework/node/init/init.go:32
Jan 8 06:51:35.075: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-network] Networking
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-network] Networking
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/08/23 06:51:35.254�[0m
�[1mSTEP:�[0m Collecting events from namespace "nettest-7775". �[38;5;243m01/08/23 06:51:35.254�[0m
�[1mSTEP:�[0m Found 40 events. �[38;5;243m01/08/23 06:51:35.432�[0m
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:46:33 +0000 UTC - event for netserver-0: {default-scheduler } Scheduled: Successfully assigned nettest-7775/netserver-0 to i-0685adc783a822be6
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:46:34 +0000 UTC - event for netserver-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "159552b9e7dd451dd6314f8e5ca347e17bf0e3abaf40525e89a55975b295abc5": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:46:34 +0000 UTC - event for netserver-1: {kubelet i-0bd196cab651da520} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:46:34 +0000 UTC - event for netserver-1: {default-scheduler } Scheduled: Successfully assigned nettest-7775/netserver-1 to i-0bd196cab651da520
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:46:34 +0000 UTC - event for netserver-2: {kubelet i-0d149644e2edcf9cf} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6155a5e6ed1080dca176729c52ef97c55e9bb7188f3fe9266dfc665c821e7cfc": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:46:34 +0000 UTC - event for netserver-2: {default-scheduler } Scheduled: Successfully assigned nettest-7775/netserver-2 to i-0d149644e2edcf9cf
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:46:34 +0000 UTC - event for netserver-3: {default-scheduler } Scheduled: Successfully assigned nettest-7775/netserver-3 to i-0f469f08be563dad4
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:46:34 +0000 UTC - event for netserver-3: {kubelet i-0f469f08be563dad4} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f51cbfe5758fb3f4aafddf0bf0bcfde14e71486f31f6d9aeb0a1bb8cc624a78f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:46:35 +0000 UTC - event for netserver-1: {kubelet i-0bd196cab651da520} Created: Created container webserver
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:46:35 +0000 UTC - event for netserver-1: {kubelet i-0bd196cab651da520} Started: Started container webserver
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:46:44 +0000 UTC - event for netserver-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b27e7ba2289e44c3df316480cd271d29b0d50c55f22385e4e990a6daa35aa595": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:46:46 +0000 UTC - event for netserver-2: {kubelet i-0d149644e2edcf9cf} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c5863b8596d945b035d0c76d7f473177b853140181e1deec2414797c28d5c24d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:46:48 +0000 UTC - event for netserver-3: {kubelet i-0f469f08be563dad4} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d74bf71c61de7a21a89a4104fc4080cae2368358c60fde3a2e06138da215a45f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:46:57 +0000 UTC - event for netserver-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "eb8f53c964255190eca82ad902342d146c4da3ce7fa4e4c2a0201da846bb42bb": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:47:02 +0000 UTC - event for netserver-2: {kubelet i-0d149644e2edcf9cf} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7f72e022a0b9b04cb4c920a2c4b5ae2fb322c5ff19f51d5d37e1727360ceb2d6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:47:03 +0000 UTC - event for netserver-3: {kubelet i-0f469f08be563dad4} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8445242caa2b47749975a8c1e8943e80accad4519abe70f2e12cff90bd2c4a2b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:47:10 +0000 UTC - event for netserver-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "af179bd53d7735043c5657556dd74d2539aa9024f135752bdf37d9e5970b4d84": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:47:13 +0000 UTC - event for netserver-2: {kubelet i-0d149644e2edcf9cf} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3d2d0a475863429fb1b4397fa0acce678528c50cac3895d74508efe0be029f46": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:47:17 +0000 UTC - event for netserver-3: {kubelet i-0f469f08be563dad4} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "05022da6a0229c6fc0261400a434edda8417e169947b557dfe379062fffe8aa2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:47:23 +0000 UTC - event for netserver-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bc61a3902039b88fd3ae7a343db79d299776fbd4793d037bc4675cab05e85e10": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:47:25 +0000 UTC - event for netserver-2: {kubelet i-0d149644e2edcf9cf} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6a4fa3703a9fed6c930fc83323e8b08144670c38e69cda9067472ba0c982e2fb": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:47:31 +0000 UTC - event for netserver-3: {kubelet i-0f469f08be563dad4} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d4d498892a59aef8a6960b9d480f046ca8db4cfbd06f48ba0d699ec3c734db3b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:47:37 +0000 UTC - event for netserver-2: {kubelet i-0d149644e2edcf9cf} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0dc56947a9820e144482aa2fcf0f676f2945bd1fbda085631193bd6ee744f7f6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:47:38 +0000 UTC - event for netserver-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f2810c40a370d06d9ba465e07f545c878b0ac91fe27cbcea7421f816aaa07d87": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:47:43 +0000 UTC - event for netserver-3: {kubelet i-0f469f08be563dad4} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "72a590cc4d6cf1d74eb5ddba4dda54dbd74abbbbef1b9a9eb6be15ab7d0e3656": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:47:48 +0000 UTC - event for netserver-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a754ec6e0fa300f369a72e89896e41b9dca3f4333687b7b5fa5149cb92ed5ae3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:47:51 +0000 UTC - event for netserver-2: {kubelet i-0d149644e2edcf9cf} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "fb969c2c0d575d6f1a5fb8070fff91dc856bd0dfa43b29caf096710e86a782fa": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:47:54 +0000 UTC - event for netserver-3: {kubelet i-0f469f08be563dad4} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7435e366880dbf7204620af97436508ea75a07a9191e1c9deddddbb6c002590e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:47:59 +0000 UTC - event for netserver-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b212972a0b104526c08fa19bf33fc4bf548c7df3f4161d709d458c93ca751fdb": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:48:04 +0000 UTC - event for netserver-2: {kubelet i-0d149644e2edcf9cf} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5881543a2172e240c0bf6ca1a5ec556f1be9d6f53229bc8639a64a40ada33582": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:48:05 +0000 UTC - event for netserver-3: {kubelet i-0f469f08be563dad4} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9ea9cd2da150c3c82b4a866185f17123ccc5aaaa2ef5052b563ab5e49caa2427": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:48:11 +0000 UTC - event for netserver-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "13e0e9de30a9676c62a8a736a39852a4ee19606fbef9e89ee324a4d1e3164a18": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:48:17 +0000 UTC - event for netserver-3: {kubelet i-0f469f08be563dad4} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c0fced28310a07cd48a933d88ad74439087462521f48051a17f784485dc9194f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:48:19 +0000 UTC - event for netserver-2: {kubelet i-0d149644e2edcf9cf} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "088a96fd7027f39de43857913f091812895dc03faf0118c6b56aa92fe6ecfe5f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:48:24 +0000 UTC - event for netserver-0: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8f948f1f3f3ef06daabbbed140f9ee90e5c47de820595ecab8a6ac8503bb15c2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:48:30 +0000 UTC - event for netserver-3: {kubelet i-0f469f08be563dad4} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4fd7572a230f1952f7f1b7802c677f51918ee90ff9ec8dc80aad0e01b522aff1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:48:32 +0000 UTC - event for netserver-2: {kubelet i-0d149644e2edcf9cf} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6790a38681cc566cd034bcd965ff72e5ed386ffcc877c65c74e7c76eb7f6ab29": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:50:55 +0000 UTC - event for netserver-2: {kubelet i-0d149644e2edcf9cf} Started: Started container webserver
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:50:55 +0000 UTC - event for netserver-2: {kubelet i-0d149644e2edcf9cf} Created: Created container webserver
Jan 8 06:51:35.432: INFO: At 2023-01-08 06:50:55 +0000 UTC - event for netserver-2: {kubelet i-0d149644e2edcf9cf} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Jan 8 06:51:35.610: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 8 06:51:35.610: INFO: netserver-0 i-0685adc783a822be6 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:33 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:33 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:33 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:33 +0000 UTC }]
Jan 8 06:51:35.610: INFO: netserver-1 i-0bd196cab651da520 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:34 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:54 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:54 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:34 +0000 UTC }]
Jan 8 06:51:35.610: INFO: netserver-2 i-0d149644e2edcf9cf Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:34 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:51:14 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:51:14 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:34 +0000 UTC }]
Jan 8 06:51:35.610: INFO: netserver-3 i-0f469f08be563dad4 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:34 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:34 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:34 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:46:34 +0000 UTC }]
Jan 8 06:51:35.610: INFO:
Jan 8 06:51:35.789: INFO: Unable to fetch nettest-7775/netserver-0/webserver logs: the server rejected our request for an unknown reason (get pods netserver-0)
Jan 8 06:51:36.329: INFO: Unable to fetch nettest-7775/netserver-3/webserver logs: the server rejected our request for an unknown reason (get pods netserver-3)
Jan 8 06:51:36.508: INFO:
Logging node info for node i-0685adc783a822be6
Jan 8 06:51:36.685: INFO: Node Info: &Node{ObjectMeta:{i-0685adc783a822be6 3eb588c7-cc76-44b4-b28a-492b9989254b 40102 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0685adc783a822be6 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0685adc783a822be6 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.112 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-8912":"i-0685adc783a822be6","ebs.csi.aws.com":"i-0685adc783a822be6"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:45:06 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-08 06:47:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0685adc783a822be6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:47:14 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:47:14 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:47:14 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:47:14 +0000 UTC,LastTransitionTime:2023-01-08 06:25:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.112,},NodeAddress{Type:ExternalIP,Address:43.201.46.149,},NodeAddress{Type:InternalDNS,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-201-46-149.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dccafbf198926b990228825ae0608,SystemUUID:ec2dccaf-bf19-8926-b990-228825ae0608,BootID:b8ff627c-af94-4730-be1d-6c7bfbfc818f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-8912^48255f14-8f20-11ed-bd4d-c24e6c6fc39f kubernetes.io/csi/ebs.csi.aws.com^vol-025c7f75403571919 kubernetes.io/csi/ebs.csi.aws.com^vol-050101682c0b7bd03 kubernetes.io/csi/ebs.csi.aws.com^vol-076ac56f18e348fec],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-050101682c0b7bd03,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-076ac56f18e348fec,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-025c7f75403571919,DevicePath:,},},Config:nil,},}
Jan 8 06:51:36.686: INFO:
Logging kubelet events for node i-0685adc783a822be6
Jan 8 06:51:36.866: INFO:
Logging pods the kubelet thinks is on node i-0685adc783a822be6
Jan 8 06:51:37.231: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:51:13 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container csi-attacher ready: false, restart count 0
Jan 8 06:51:37.231: INFO: inline-volume-tester-g992k started at 2023-01-08 06:44:18 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container csi-volume-tester ready: true, restart count 0
Jan 8 06:51:37.231: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-svbp2 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:37.231: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mpw58 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:37.231: INFO: csi-mockplugin-0 started at 2023-01-08 06:45:32 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container csi-provisioner ready: true, restart count 0
Jan 8 06:51:37.231: INFO: Container driver-registrar ready: true, restart count 0
Jan 8 06:51:37.231: INFO: Container mock ready: true, restart count 0
Jan 8 06:51:37.231: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rrmzm started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0
Jan 8 06:51:37.231: INFO: busybox1 started at 2023-01-08 06:51:29 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container busybox ready: false, restart count 0
Jan 8 06:51:37.231: INFO: pod-subpath-test-preprovisionedpv-hnfm started at 2023-01-08 06:46:52 +0000 UTC (2+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Init container init-volume-preprovisionedpv-hnfm ready: false, restart count 0
Jan 8 06:51:37.231: INFO: Init container test-init-volume-preprovisionedpv-hnfm ready: false, restart count 0
Jan 8 06:51:37.231: INFO: Container test-container-subpath-preprovisionedpv-hnfm ready: false, restart count 0
Jan 8 06:51:37.231: INFO: annotationupdatef61ea514-d6d7-4d4c-b02d-07adce5876f7 started at 2023-01-08 06:50:08 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container client-container ready: false, restart count 0
Jan 8 06:51:37.231: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:50:39 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container csi-attacher ready: false, restart count 0
Jan 8 06:51:37.231: INFO: pod-disruption-failure-ignore-0-w4rvn started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container c ready: false, restart count 0
Jan 8 06:51:37.231: INFO: netserver-0 started at 2023-01-08 06:46:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container webserver ready: false, restart count 0
Jan 8 06:51:37.231: INFO: hostexec-i-0685adc783a822be6-r5lld started at 2023-01-08 06:46:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container agnhost-container ready: true, restart count 0
Jan 8 06:51:37.231: INFO: ebs-csi-node-ft7bw started at 2023-01-08 06:25:38 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:51:37.231: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:51:37.231: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 8 06:51:37.231: INFO: ss-0 started at 2023-01-08 06:44:55 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container webserver ready: false, restart count 0
Jan 8 06:51:37.231: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-q97zd started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:37.231: INFO: pod1 started at 2023-01-08 06:50:04 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container agnhost-container ready: false, restart count 0
Jan 8 06:51:37.231: INFO: service-proxy-disabled-6pp8x started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container service-proxy-disabled ready: true, restart count 0
Jan 8 06:51:37.231: INFO: pod-8fb38c56-a909-49a3-85f0-9d73b5275577 started at 2023-01-08 06:50:50 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container test-container ready: false, restart count 0
Jan 8 06:51:37.231: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2bn4h started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:37.231: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v9tjx started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:37.231: INFO: csi-mockplugin-0 started at 2023-01-08 06:51:13 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container csi-provisioner ready: false, restart count 0
Jan 8 06:51:37.231: INFO: Container driver-registrar ready: false, restart count 0
Jan 8 06:51:37.231: INFO: Container mock ready: false, restart count 0
Jan 8 06:51:37.231: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-97qp6 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:37.231: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-4d46c started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:37.231: INFO: inline-volume-tester2-lv7ch started at 2023-01-08 06:44:52 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 8 06:51:37.231: INFO: pvc-volume-tester-58sz7 started at 2023-01-08 06:47:13 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container volume-tester ready: false, restart count 0
Jan 8 06:51:37.231: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-zgmck started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:37.231: INFO: cilium-9vhbx started at 2023-01-08 06:25:38 +0000 UTC (1+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 8 06:51:37.231: INFO: Container cilium-agent ready: true, restart count 0
Jan 8 06:51:37.231: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:45:32 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container csi-attacher ready: false, restart count 0
Jan 8 06:51:37.231: INFO: pod-disruption-failure-ignore-1-kfqqr started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container c ready: false, restart count 0
Jan 8 06:51:37.231: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-gq5pr started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:37.231: INFO: service-headless-bsxnn started at 2023-01-08 06:50:25 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container service-headless ready: false, restart count 0
Jan 8 06:51:37.231: INFO: busybox-user-65534-f67e52a6-8f93-482e-8c32-0b884e120810 started at 2023-01-08 06:46:55 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container busybox-user-65534-f67e52a6-8f93-482e-8c32-0b884e120810 ready: false, restart count 0
Jan 8 06:51:37.231: INFO: csi-mockplugin-0 started at 2023-01-08 06:50:39 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:51:37.231: INFO: Container csi-provisioner ready: false, restart count 0
Jan 8 06:51:37.231: INFO: Container driver-registrar ready: false, restart count 0
Jan 8 06:51:37.231: INFO: Container mock ready: false, restart count 0
Jan 8 06:51:38.003: INFO:
Latency metrics for node i-0685adc783a822be6
Jan 8 06:51:38.003: INFO:
Logging node info for node i-0bd196cab651da520
Jan 8 06:51:38.184: INFO: Node Info: &Node{ObjectMeta:{i-0bd196cab651da520 f7f00780-d24e-4c8e-b54d-373598e1d1d2 41430 0 2023-01-08 06:26:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0bd196cab651da520 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0bd196cab651da520 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.104 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bd196cab651da520"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:26:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:50:48 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-08 06:50:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0bd196cab651da520,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:50:54 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:50:54 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:50:54 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:50:54 +0000 UTC,LastTransitionTime:2023-01-08 06:26:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.104,},NodeAddress{Type:ExternalIP,Address:54.180.1.55,},NodeAddress{Type:InternalDNS,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-180-1-55.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec218f9d32a84b2feeb6301255fd576c,SystemUUID:ec218f9d-32a8-4b2f-eeb6-301255fd576c,BootID:accdf7fd-1195-4765-b28e-29d44bf47bac,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-021c1df2b6aa5a7be kubernetes.io/csi/ebs.csi.aws.com^vol-02d83118eb07c1244 kubernetes.io/csi/ebs.csi.aws.com^vol-06c6e766891904b0f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-06c6e766891904b0f,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-021c1df2b6aa5a7be,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-02d83118eb07c1244,DevicePath:,},},Config:nil,},}
Jan 8 06:51:38.185: INFO:
Logging kubelet events for node i-0bd196cab651da520
Jan 8 06:51:38.365: INFO:
Logging pods the kubelet thinks is on node i-0bd196cab651da520
Jan 8 06:51:38.550: INFO: hostexec-i-0bd196cab651da520-lk7nk started at 2023-01-08 06:50:41 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container agnhost-container ready: true, restart count 0
Jan 8 06:51:38.550: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-76nxw started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:38.550: INFO: pod-a15c4150-c88e-49b1-a111-7f2271cf819c started at 2023-01-08 06:46:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container write-pod ready: false, restart count 0
Jan 8 06:51:38.550: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v44bk started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:38.550: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-75r8w started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:38.550: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9mp69 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:38.550: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-cddx5 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:38.550: INFO: netserver-1 started at 2023-01-08 06:46:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container webserver ready: true, restart count 0
Jan 8 06:51:38.550: INFO: security-context-6fa8ddf8-7350-44bd-841b-327e9630ec84 started at 2023-01-08 06:51:09 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container test-container ready: false, restart count 0
Jan 8 06:51:38.550: INFO: pod-fea82a8a-61d5-4edb-b62d-f998d68a0314 started at 2023-01-08 06:50:46 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container write-pod ready: false, restart count 0
Jan 8 06:51:38.550: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-6df8l started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:38.550: INFO: pod-projected-configmaps-1b1db2bb-9759-4536-97eb-e0b261ef4c1b started at 2023-01-08 06:47:28 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container agnhost-container ready: false, restart count 0
Jan 8 06:51:38.550: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-vrcjq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:38.550: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9776c started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:38.550: INFO: inline-volume-tester-hgbc6 started at 2023-01-08 06:50:45 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container csi-volume-tester ready: false, restart count 0
Jan 8 06:51:38.550: INFO: webserver-pod started at 2023-01-08 06:46:47 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container agnhost ready: false, restart count 0
Jan 8 06:51:38.550: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9wskq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:38.550: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mphgx started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:38.550: INFO: oidc-discovery-validator started at 2023-01-08 06:51:00 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container oidc-discovery-validator ready: false, restart count 0
Jan 8 06:51:38.550: INFO: pod1 started at 2023-01-08 06:51:10 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container agnhost-container ready: false, restart count 0
Jan 8 06:51:38.550: INFO: cilium-s6shx started at 2023-01-08 06:26:02 +0000 UTC (1+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 8 06:51:38.550: INFO: Container cilium-agent ready: true, restart count 0
Jan 8 06:51:38.550: INFO: ebs-csi-node-6grlm started at 2023-01-08 06:26:02 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:51:38.550: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:51:38.550: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 8 06:51:38.550: INFO: service-headless-x4nfw started at 2023-01-08 06:50:25 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:38.550: INFO: Container service-headless ready: false, restart count 0
Jan 8 06:51:39.232: INFO:
Latency metrics for node i-0bd196cab651da520
Jan 8 06:51:39.232: INFO:
Logging node info for node i-0d149644e2edcf9cf
Jan 8 06:51:39.410: INFO: Node Info: &Node{ObjectMeta:{i-0d149644e2edcf9cf b448ef72-6e61-4ad5-aa29-d63cc0f94945 41768 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0d149644e2edcf9cf kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0d149644e2edcf9cf topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.115 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0d149644e2edcf9cf"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:51:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0d149644e2edcf9cf,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:51:18 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:51:18 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:51:18 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:51:18 +0000 UTC,LastTransitionTime:2023-01-08 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.115,},NodeAddress{Type:ExternalIP,Address:3.35.219.218,},NodeAddress{Type:InternalDNS,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-35-219-218.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec26c6a9af66729e278545162595eb2d,SystemUUID:ec26c6a9-af66-729e-2785-45162595eb2d,BootID:959da9e3-fb86-442a-958f-2a49c509cdf4,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 8 06:51:39.410: INFO:
Logging kubelet events for node i-0d149644e2edcf9cf
Jan 8 06:51:39.590: INFO:
Logging pods the kubelet thinks is on node i-0d149644e2edcf9cf
Jan 8 06:51:39.774: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9b2xq started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:39.774: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:39.774: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-njh2f started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:39.774: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:39.774: INFO: cilium-58twl started at 2023-01-08 06:25:37 +0000 UTC (1+1 container statuses recorded)
Jan 8 06:51:39.774: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 8 06:51:39.774: INFO: Container cilium-agent ready: true, restart count 0
Jan 8 06:51:39.774: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rv5qq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:39.774: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:39.774: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-flhps started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:39.774: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:39.774: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2vh6v started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:39.774: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0
Jan 8 06:51:39.774: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nt9jr started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:39.774: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0
Jan 8 06:51:39.774: INFO: ebs-csi-node-lfcwb started at 2023-01-08 06:25:37 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:51:39.774: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:51:39.774: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:51:39.774: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 8 06:51:39.774: INFO: coredns-559769c974-9q4qg started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:39.774: INFO: Container coredns ready: true, restart count 0
Jan 8 06:51:39.774: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-xrmq4 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:39.774: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:39.774: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-7gwsq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:39.774: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:39.774: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nsfnm started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:39.774: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:39.774: INFO: netserver-2 started at 2023-01-08 06:46:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:39.774: INFO: Container webserver ready: true, restart count 0
Jan 8 06:51:39.774: INFO: coredns-autoscaler-7cb5c5b969-xqnk5 started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:39.774: INFO: Container autoscaler ready: true, restart count 0
Jan 8 06:51:39.774: INFO: service-proxy-disabled-v5f82 started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:39.774: INFO: Container service-proxy-disabled ready: true, restart count 0
Jan 8 06:51:39.774: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-tgn6r started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:39.774: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:40.448: INFO:
Latency metrics for node i-0d149644e2edcf9cf
Jan 8 06:51:40.448: INFO:
Logging node info for node i-0eee6113fd9f81d94
Jan 8 06:51:40.627: INFO: Node Info: &Node{ObjectMeta:{i-0eee6113fd9f81d94 6de592fc-f11c-4ba3-afa0-707bb4ca29b2 39949 0 2023-01-08 06:23:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0eee6113fd9f81d94 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.48.64 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eee6113fd9f81d94"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-08 06:23:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-08 06:24:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-08 06:24:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-08 06:46:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0eee6113fd9f81d94,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892301824 0} {<nil>} 3801076Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787444224 0} {<nil>} 3698676Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:46:48 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:46:48 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:46:48 +0000 UTC,LastTransitionTime:2023-01-08 06:23:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:46:48 +0000 UTC,LastTransitionTime:2023-01-08 06:24:35 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.48.64,},NodeAddress{Type:ExternalIP,Address:3.35.206.95,},NodeAddress{Type:InternalDNS,Address:i-0eee6113fd9f81d94.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0eee6113fd9f81d94.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-35-206-95.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2d8cf21861b15bb593210ff59f6044,SystemUUID:ec2d8cf2-1861-b15b-b593-210ff59f6044,BootID:fcad947c-a63f-4722-9a63-22eac74bea9f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:66a453db625abb268f4b3bbefc5a34a171d81e6e8796cecca54cfd71775c77c4 registry.k8s.io/etcdadm/etcd-manager:v3.0.20221209],SizeBytes:231502799,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.0],SizeBytes:135162323,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.0],SizeBytes:124991801,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.0],SizeBytes:57657656,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:43453935,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42803460,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4967354,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 8 06:51:40.627: INFO:
Logging kubelet events for node i-0eee6113fd9f81d94
Jan 8 06:51:40.807: INFO:
Logging pods the kubelet thinks is on node i-0eee6113fd9f81d94
Jan 8 06:51:40.990: INFO: kube-controller-manager-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:40.990: INFO: Container kube-controller-manager ready: true, restart count 2
Jan 8 06:51:40.990: INFO: ebs-csi-node-nspqr started at 2023-01-08 06:24:21 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:51:40.990: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:51:40.990: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:51:40.990: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 8 06:51:40.990: INFO: aws-cloud-controller-manager-d8d56 started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:40.990: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 8 06:51:40.990: INFO: cilium-operator-5dd44dc49f-fd8vl started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:40.990: INFO: Container cilium-operator ready: true, restart count 0
Jan 8 06:51:40.990: INFO: ebs-csi-controller-89b649c94-vzzj9 started at 2023-01-08 06:24:21 +0000 UTC (0+5 container statuses recorded)
Jan 8 06:51:40.990: INFO: Container csi-attacher ready: true, restart count 0
Jan 8 06:51:40.990: INFO: Container csi-provisioner ready: true, restart count 0
Jan 8 06:51:40.990: INFO: Container csi-resizer ready: true, restart count 0
Jan 8 06:51:40.990: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:51:40.990: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:51:40.990: INFO: dns-controller-69987775c6-v4cmg started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:40.990: INFO: Container dns-controller ready: true, restart count 0
Jan 8 06:51:40.990: INFO: kube-scheduler-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:40.990: INFO: Container kube-scheduler ready: true, restart count 0
Jan 8 06:51:40.990: INFO: kube-apiserver-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+2 container statuses recorded)
Jan 8 06:51:40.990: INFO: Container healthcheck ready: true, restart count 0
Jan 8 06:51:40.990: INFO: Container kube-apiserver ready: true, restart count 1
Jan 8 06:51:40.990: INFO: kops-controller-242qq started at 2023-01-08 06:24:21 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:40.990: INFO: Container kops-controller ready: true, restart count 0
Jan 8 06:51:40.990: INFO: cilium-f22px started at 2023-01-08 06:24:21 +0000 UTC (1+1 container statuses recorded)
Jan 8 06:51:40.990: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 8 06:51:40.990: INFO: Container cilium-agent ready: true, restart count 0
Jan 8 06:51:40.990: INFO: etcd-manager-events-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:40.990: INFO: Container etcd-manager ready: true, restart count 0
Jan 8 06:51:40.990: INFO: etcd-manager-main-i-0eee6113fd9f81d94 started at 2023-01-08 06:23:14 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:40.990: INFO: Container etcd-manager ready: true, restart count 0
Jan 8 06:51:41.589: INFO:
Latency metrics for node i-0eee6113fd9f81d94
Jan 8 06:51:41.589: INFO:
Logging node info for node i-0f469f08be563dad4
Jan 8 06:51:41.767: INFO: Node Info: &Node{ObjectMeta:{i-0f469f08be563dad4 fddb222c-a6ac-415f-83dc-4b71acb0e5ab 41025 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0f469f08be563dad4 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0f469f08be563dad4 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.57.173 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0f469f08be563dad4"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:33 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-08 06:50:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0f469f08be563dad4,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:49:59 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:49:59 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:49:59 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:49:59 +0000 UTC,LastTransitionTime:2023-01-08 06:25:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.57.173,},NodeAddress{Type:ExternalIP,Address:3.38.152.241,},NodeAddress{Type:InternalDNS,Address:i-0f469f08be563dad4.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f469f08be563dad4.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-38-152-241.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2abc7b1d816532cfb4787d85cb98ea,SystemUUID:ec2abc7b-1d81-6532-cfb4-787d85cb98ea,BootID:442daa6f-46d1-4c32-9182-8b6204c22aba,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 8 06:51:41.768: INFO:
Logging kubelet events for node i-0f469f08be563dad4
Jan 8 06:51:41.953: INFO:
Logging pods the kubelet thinks is on node i-0f469f08be563dad4
Jan 8 06:51:42.136: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-h5c5c started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:42.136: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-jc58p started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:42.136: INFO: service-headless-bn7ft started at 2023-01-08 06:50:25 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container service-headless ready: true, restart count 0
Jan 8 06:51:42.136: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-gdlbp started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:42.136: INFO: service-proxy-disabled-bqbxv started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container service-proxy-disabled ready: false, restart count 0
Jan 8 06:51:42.136: INFO: pause-pod-1 started at 2023-01-08 06:46:50 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container agnhost-container ready: true, restart count 0
Jan 8 06:51:42.136: INFO: netserver-3 started at 2023-01-08 06:46:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container webserver ready: false, restart count 0
Jan 8 06:51:42.136: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-glvzd started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:42.136: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-cwg4t started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:42.136: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-57rwf started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0
Jan 8 06:51:42.136: INFO: coredns-559769c974-f9s4x started at 2023-01-08 06:26:19 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container coredns ready: true, restart count 0
Jan 8 06:51:42.136: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-b6m8x started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:42.136: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-ktkbb started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:42.136: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-dvpgx started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:42.136: INFO: cilium-24rbm started at 2023-01-08 06:25:37 +0000 UTC (1+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 8 06:51:42.136: INFO: Container cilium-agent ready: true, restart count 0
Jan 8 06:51:42.136: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-x88cb started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0
Jan 8 06:51:42.136: INFO: ebs-csi-node-9s8qc started at 2023-01-08 06:25:37 +0000 UTC (0+3 container statuses recorded)
Jan 8 06:51:42.136: INFO: Container ebs-plugin ready: true, restart count 0
Jan 8 06:51:42.136: INFO: Container liveness-probe ready: true, restart count 0
Jan 8 06:51:42.136: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 8 06:51:42.763: INFO:
Latency metrics for node i-0f469f08be563dad4
[DeferCleanup (Each)] [sig-network] Networking
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "nettest-7775" for this suite. �[38;5;243m01/08/23 06:51:42.763�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sProxy\sversion\sv1\sA\sset\sof\svalid\sresponses\sare\sreturned\sfor\sboth\spod\sand\sservice\sProxyWithPath\s\[Conformance\]$'
test/e2e/network/proxy.go:315 k8s.io/kubernetes/test/e2e/network.glob..func25.1.4() test/e2e/network/proxy.go:315 +0x61d
[BeforeEach] version v1 set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/08/23 06:45:56.635�[0m Jan 8 06:45:56.635: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename proxy �[38;5;243m01/08/23 06:45:56.636�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/08/23 06:45:57.169�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/08/23 06:45:57.522�[0m [BeforeEach] version v1 test/e2e/framework/metrics/init/init.go:31 [It] A set of valid responses are returned for both pod and service ProxyWithPath [Conformance] test/e2e/network/proxy.go:286 Jan 8 06:45:57.875: INFO: Creating pod... Jan 8 06:45:58.056: INFO: Waiting up to 5m0s for pod "agnhost" in namespace "proxy-2863" to be "running" Jan 8 06:45:58.233: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 177.539266ms Jan 8 06:46:00.412: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2.356012044s Jan 8 06:46:02.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4.355509283s Jan 8 06:46:04.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 6.355045491s Jan 8 06:46:06.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 8.355899288s Jan 8 06:46:08.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 10.355033296s Jan 8 06:46:10.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 12.35496544s Jan 8 06:46:12.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 14.355719787s Jan 8 06:46:14.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 16.355350067s Jan 8 06:46:16.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 18.354902763s Jan 8 06:46:18.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 20.355031778s Jan 8 06:46:20.412: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 22.356034306s Jan 8 06:46:22.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 24.355438226s Jan 8 06:46:24.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 26.355434405s Jan 8 06:46:26.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 28.355118774s Jan 8 06:46:28.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 30.355816049s Jan 8 06:46:30.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 32.355838464s Jan 8 06:46:32.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 34.355767907s Jan 8 06:46:34.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 36.355253866s Jan 8 06:46:36.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 38.35569923s Jan 8 06:46:38.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 40.355304542s Jan 8 06:46:40.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 42.354859152s Jan 8 06:46:42.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 44.355423816s Jan 8 06:46:44.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 46.354845889s Jan 8 06:46:46.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 48.355597585s Jan 8 06:46:48.413: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 50.35730126s Jan 8 06:46:50.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 52.35542242s Jan 8 06:46:52.412: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 54.356095276s Jan 8 06:46:54.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 56.35556723s Jan 8 06:46:56.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 58.355251933s Jan 8 06:46:58.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.355008969s Jan 8 06:47:00.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.354961884s Jan 8 06:47:02.412: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.355975663s Jan 8 06:47:04.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.355249156s Jan 8 06:47:06.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.355167521s Jan 8 06:47:08.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.355085579s Jan 8 06:47:10.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.355550255s Jan 8 06:47:12.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.355153746s Jan 8 06:47:14.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.354825857s Jan 8 06:47:16.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.35507275s Jan 8 06:47:18.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.355818178s Jan 8 06:47:20.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.354928292s Jan 8 06:47:22.416: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.360467087s Jan 8 06:47:24.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.355723499s Jan 8 06:47:26.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.354665113s Jan 8 06:47:28.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.355885569s Jan 8 06:47:30.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.355447223s Jan 8 06:47:32.412: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.356012206s Jan 8 06:47:34.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.355434897s Jan 8 06:47:36.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.355115622s Jan 8 06:47:38.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.354986596s Jan 8 06:47:40.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.355064941s Jan 8 06:47:42.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.355652684s Jan 8 06:47:44.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.355332098s Jan 8 06:47:46.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.355194036s Jan 8 06:47:48.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.35528601s Jan 8 06:47:50.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.354927575s Jan 8 06:47:52.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.354999495s Jan 8 06:47:54.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.355150791s Jan 8 06:47:56.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.354712876s Jan 8 06:47:58.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.355658963s Jan 8 06:48:00.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.355268916s Jan 8 06:48:02.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.355147281s Jan 8 06:48:04.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.355218284s Jan 8 06:48:06.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.355378298s Jan 8 06:48:08.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.355099175s Jan 8 06:48:10.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.354968128s Jan 8 06:48:12.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.35516906s Jan 8 06:48:14.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.355137961s Jan 8 06:48:16.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.354940329s Jan 8 06:48:18.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.355116798s Jan 8 06:48:20.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.355897635s Jan 8 06:48:22.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.355633417s Jan 8 06:48:24.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.355179652s Jan 8 06:48:26.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.354790806s Jan 8 06:48:28.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.355071749s Jan 8 06:48:30.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.355385672s Jan 8 06:48:32.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.355284172s Jan 8 06:48:34.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.355075762s Jan 8 06:48:36.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.354854381s Jan 8 06:48:38.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.355463784s Jan 8 06:48:40.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.355192671s Jan 8 06:48:42.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.355268451s Jan 8 06:48:44.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.355034296s Jan 8 06:48:46.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.355773708s Jan 8 06:48:48.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.35530197s Jan 8 06:48:50.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.355091414s Jan 8 06:48:52.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.354880911s Jan 8 06:48:54.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.355589514s Jan 8 06:48:56.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.355482835s Jan 8 06:48:58.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.35566578s Jan 8 06:49:00.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.354980226s Jan 8 06:49:02.412: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.356061594s Jan 8 06:49:04.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.355606369s Jan 8 06:49:06.412: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.35602651s Jan 8 06:49:08.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.354770396s Jan 8 06:49:10.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.355260497s Jan 8 06:49:12.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.354929483s Jan 8 06:49:14.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.355026842s Jan 8 06:49:16.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.354924316s Jan 8 06:49:18.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.355124713s Jan 8 06:49:20.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.354961041s Jan 8 06:49:22.412: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.356226424s Jan 8 06:49:24.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.355202575s Jan 8 06:49:26.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.355306471s Jan 8 06:49:28.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.355248886s Jan 8 06:49:30.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.355495049s Jan 8 06:49:32.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.355244289s Jan 8 06:49:34.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.354942111s Jan 8 06:49:36.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.355060781s Jan 8 06:49:38.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.354838174s Jan 8 06:49:40.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.354827744s Jan 8 06:49:42.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.355568778s Jan 8 06:49:44.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.355146999s Jan 8 06:49:46.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.354475361s Jan 8 06:49:48.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.354792263s Jan 8 06:49:50.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.355393678s Jan 8 06:49:52.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.355085648s Jan 8 06:49:54.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.355216341s Jan 8 06:49:56.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.355082531s Jan 8 06:49:58.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.355080542s Jan 8 06:50:00.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.355171615s Jan 8 06:50:02.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.355014715s Jan 8 06:50:04.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.355301707s Jan 8 06:50:06.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.355390257s Jan 8 06:50:08.412: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.356584114s Jan 8 06:50:10.412: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.35621514s Jan 8 06:50:12.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.355474478s Jan 8 06:50:14.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.355421326s Jan 8 06:50:16.412: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.356006983s Jan 8 06:50:18.412: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.356007571s Jan 8 06:50:20.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.355201132s Jan 8 06:50:22.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.355302031s Jan 8 06:50:24.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.355111219s Jan 8 06:50:26.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.355948592s Jan 8 06:50:28.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.354809372s Jan 8 06:50:30.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.355948646s Jan 8 06:50:32.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.355692446s Jan 8 06:50:34.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.35542969s Jan 8 06:50:36.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.355158438s Jan 8 06:50:38.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.355165609s Jan 8 06:50:40.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.355102477s Jan 8 06:50:42.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.355580657s Jan 8 06:50:44.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.355204802s Jan 8 06:50:46.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.355167633s Jan 8 06:50:48.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.354736138s Jan 8 06:50:50.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.355314305s Jan 8 06:50:52.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.355658618s Jan 8 06:50:54.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.355616074s Jan 8 06:50:56.411: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.355286128s Jan 8 06:50:58.410: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.354798108s Jan 8 06:50:58.587: INFO: Pod "agnhost": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.531674986s Jan 8 06:50:58.589: INFO: Unexpected error: Pod didn't start within time out period: <*pod.timeoutError | 0xc00152f6e0>: { msg: "timed out while waiting for pod proxy-2863/agnhost to be running", observedObjects: [ <*v1.Pod | 0xc003707680>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "agnhost", GenerateName: "", Namespace: "proxy-2863", SelfLink: "", UID: "185e530d-2cb5-4b4d-b0b1-c359f1e6b283", ResourceVersion: "39382", Generation: 0, CreationTimestamp: { Time: { wall: 0, ext: 63808757157, loc: { name: "Local", zone: [ {name: "UTC", offset: 0, isDST: false}, ], tx: [ { when: -576460752303423488, index: 0, isstd: false, isutc: false, }, ], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: "UTC", offset: 0, isDST: false}, }, }, }, DeletionTimestamp: nil, DeletionGracePeriodSeconds: nil, Labels: {"test": "response"}, Annotations: nil, OwnerReferences: nil, Finalizers: nil, ManagedFields: [ { Manager: "e2e.test", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63808757157, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:metadata\":{\"f:labels\":{\".\":{},\"f:test\":{}}},\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"agnhost\\\"}\":{\".\":{},\"f:command\":{},\"f:env\":{\".\":{},\"k:{\\\"name\\\":\\\"SERVE_PORT_80\\\"}\":{\".\":{},\"f:name\":{},\"f:value\":{}}},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:terminationGracePeriodSeconds\":{}}}", }, Subresource: "", }, { Manager: "kubelet", Operation: "Update", APIVersion: "v1", Time: { Time: { wall: 0, ext: 63808757157, loc: { name: "Local", zone: [...], tx: [...], extend: "UTC0", cacheStart: 9223372036854775807, cacheEnd: 9223372036854775807, cacheZone: {name: ..., offset: ..., isDST: ...}, }, }, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}", }, Subresource: "status", }, ], }, Spec: { Volumes: [ { Name: "kube-api-access-fcmgw", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: nil, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: { Sources: [ { Secret: ..., DownwardAPI: ..., ConfigMap: ..., ServiceAccountToken: ..., }, { Secret: ..., DownwardAPI: ..., ConfigMap: ..., ServiceAccountToken: ..., }, { Secret: ..., DownwardAPI: ..., ConfigMap: ..., ServiceAccountToken: ..., }, ], DefaultMode: 420, }, ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Jan 8 06:50:58.589: FAIL: Pod didn't start within time out period: timed out while waiting for pod proxy-2863/agnhost to be running Full Stack Trace k8s.io/kubernetes/test/e2e/network.glob..func25.1.4() test/e2e/network/proxy.go:315 +0x61d [AfterEach] version v1 test/e2e/framework/node/init/init.go:32 Jan 8 06:50:58.589: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] version v1 test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] version v1 dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/08/23 06:50:58.768�[0m �[1mSTEP:�[0m Collecting events from namespace "proxy-2863". �[38;5;243m01/08/23 06:50:58.768�[0m �[1mSTEP:�[0m Found 11 events. �[38;5;243m01/08/23 06:50:58.945�[0m Jan 8 06:50:58.945: INFO: At 2023-01-08 06:45:57 +0000 UTC - event for agnhost: {default-scheduler } Scheduled: Successfully assigned proxy-2863/agnhost to i-0685adc783a822be6 Jan 8 06:50:58.945: INFO: At 2023-01-08 06:45:58 +0000 UTC - event for agnhost: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "104aedef9dc39f9f2459c7382f0ea9b57091dca45889f33370355da55fc673bf": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:50:58.945: INFO: At 2023-01-08 06:46:08 +0000 UTC - event for agnhost: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "20f11dcbd70f896cb402789fb45be83fafbbf343acd85d93c618d8f2cb5db929": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:50:58.945: INFO: At 2023-01-08 06:46:23 +0000 UTC - event for agnhost: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "eeb451f2a1e5ac1b8d045300f35e162304f2bdd8edc7591ec40bf5ed988680d0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:50:58.945: INFO: At 2023-01-08 06:46:36 +0000 UTC - event for agnhost: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "34ece852c564bbcf54178fd21dc823f341356e9466bda25e8ae524be1eb04e29": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:50:58.945: INFO: At 2023-01-08 06:46:49 +0000 UTC - event for agnhost: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5dc95863c242b02dd97ae13086b8b634d38789abaff72535b0c626c4f2447c3f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:50:58.945: INFO: At 2023-01-08 06:47:02 +0000 UTC - event for agnhost: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0d5b609af538c9ad2a51b9bcd989b4021c7a6049b8027049b85b42941c9cf407": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:50:58.945: INFO: At 2023-01-08 06:47:17 +0000 UTC - event for agnhost: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7f079f194042e3c5a5e38e176d87280ff2819c0b9e23c69ea642ba029d37f436": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:50:58.945: INFO: At 2023-01-08 06:47:31 +0000 UTC - event for agnhost: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f9ad0f9e85f736b191e6dae5de385e3f51589931e7629c0d973bb06863473437": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:50:58.945: INFO: At 2023-01-08 06:47:45 +0000 UTC - event for agnhost: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "2be9e5fa02a261a220381bb0ffa173a3ba8ae6f3c86f086ac62a3ad6c754e4d3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:50:58.945: INFO: At 2023-01-08 06:47:56 +0000 UTC - event for agnhost: {kubelet i-0685adc783a822be6} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0390e9ed358fb0e622c8166377999a6abf5a67e889cebe28dcad0fc9e4f1a1b4": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 8 06:50:59.123: INFO: POD NODE PHASE GRACE CONDITIONS Jan 8 06:50:59.123: INFO: agnhost i-0685adc783a822be6 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:45:57 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:45:57 +0000 UTC ContainersNotReady containers with unready status: [agnhost]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:45:57 +0000 UTC ContainersNotReady containers with unready status: [agnhost]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-08 06:45:57 +0000 UTC }] Jan 8 06:50:59.123: INFO: Jan 8 06:50:59.301: INFO: Unable to fetch proxy-2863/agnhost/agnhost logs: the server rejected our request for an unknown reason (get pods agnhost) Jan 8 06:50:59.479: INFO: Logging node info for node i-0685adc783a822be6 Jan 8 06:50:59.656: INFO: Node Info: &Node{ObjectMeta:{i-0685adc783a822be6 3eb588c7-cc76-44b4-b28a-492b9989254b 40102 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0685adc783a822be6 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0685adc783a822be6 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.40.112 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-8912":"i-0685adc783a822be6","ebs.csi.aws.com":"i-0685adc783a822be6"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:45:06 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-08 06:47:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0685adc783a822be6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:47:14 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:47:14 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:47:14 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:47:14 +0000 UTC,LastTransitionTime:2023-01-08 06:25:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.112,},NodeAddress{Type:ExternalIP,Address:43.201.46.149,},NodeAddress{Type:InternalDNS,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0685adc783a822be6.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-43-201-46-149.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dccafbf198926b990228825ae0608,SystemUUID:ec2dccaf-bf19-8926-b990-228825ae0608,BootID:b8ff627c-af94-4730-be1d-6c7bfbfc818f,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-8912^48255f14-8f20-11ed-bd4d-c24e6c6fc39f kubernetes.io/csi/ebs.csi.aws.com^vol-025c7f75403571919 kubernetes.io/csi/ebs.csi.aws.com^vol-050101682c0b7bd03 kubernetes.io/csi/ebs.csi.aws.com^vol-076ac56f18e348fec],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-050101682c0b7bd03,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-076ac56f18e348fec,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-025c7f75403571919,DevicePath:,},},Config:nil,},} Jan 8 06:50:59.656: INFO: Logging kubelet events for node i-0685adc783a822be6 Jan 8 06:50:59.835: INFO: Logging pods the kubelet thinks is on node i-0685adc783a822be6 Jan 8 06:51:00.024: INFO: pvc-volume-tester-58sz7 started at 2023-01-08 06:47:13 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container volume-tester ready: false, restart count 0 Jan 8 06:51:00.024: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-zgmck started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:00.024: INFO: pod-disruption-failure-ignore-1-kfqqr started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container c ready: false, restart count 0 Jan 8 06:51:00.024: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-gq5pr started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:00.024: INFO: service-headless-bsxnn started at 2023-01-08 06:50:25 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container service-headless ready: false, restart count 0 Jan 8 06:51:00.024: INFO: busybox-user-65534-f67e52a6-8f93-482e-8c32-0b884e120810 started at 2023-01-08 06:46:55 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container busybox-user-65534-f67e52a6-8f93-482e-8c32-0b884e120810 ready: false, restart count 0 Jan 8 06:51:00.024: INFO: csi-mockplugin-0 started at 2023-01-08 06:50:39 +0000 UTC (0+3 container statuses recorded) Jan 8 06:51:00.024: INFO: Container csi-provisioner ready: false, restart count 0 Jan 8 06:51:00.024: INFO: Container driver-registrar ready: false, restart count 0 Jan 8 06:51:00.024: INFO: Container mock ready: false, restart count 0 Jan 8 06:51:00.024: INFO: cilium-9vhbx started at 2023-01-08 06:25:38 +0000 UTC (1+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:51:00.024: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:51:00.024: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:45:32 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container csi-attacher ready: false, restart count 0 Jan 8 06:51:00.024: INFO: inline-volume-tester-g992k started at 2023-01-08 06:44:18 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container csi-volume-tester ready: true, restart count 0 Jan 8 06:51:00.024: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-svbp2 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:00.024: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mpw58 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:00.024: INFO: csi-mockplugin-0 started at 2023-01-08 06:45:32 +0000 UTC (0+3 container statuses recorded) Jan 8 06:51:00.024: INFO: Container csi-provisioner ready: true, restart count 0 Jan 8 06:51:00.024: INFO: Container driver-registrar ready: true, restart count 0 Jan 8 06:51:00.024: INFO: Container mock ready: true, restart count 0 Jan 8 06:51:00.024: INFO: pod-subpath-test-preprovisionedpv-hnfm started at 2023-01-08 06:46:52 +0000 UTC (2+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Init container init-volume-preprovisionedpv-hnfm ready: false, restart count 0 Jan 8 06:51:00.024: INFO: Init container test-init-volume-preprovisionedpv-hnfm ready: false, restart count 0 Jan 8 06:51:00.024: INFO: Container test-container-subpath-preprovisionedpv-hnfm ready: false, restart count 0 Jan 8 06:51:00.024: INFO: annotationupdatef61ea514-d6d7-4d4c-b02d-07adce5876f7 started at 2023-01-08 06:50:08 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container client-container ready: false, restart count 0 Jan 8 06:51:00.024: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rrmzm started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0 Jan 8 06:51:00.024: INFO: agnhost started at 2023-01-08 06:45:57 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container agnhost ready: false, restart count 0 Jan 8 06:51:00.024: INFO: netserver-0 started at 2023-01-08 06:46:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container webserver ready: false, restart count 0 Jan 8 06:51:00.024: INFO: hostexec-i-0685adc783a822be6-r5lld started at 2023-01-08 06:46:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container agnhost-container ready: true, restart count 0 Jan 8 06:51:00.024: INFO: csi-mockplugin-attacher-0 started at 2023-01-08 06:50:39 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container csi-attacher ready: false, restart count 0 Jan 8 06:51:00.024: INFO: pod-disruption-failure-ignore-0-w4rvn started at 2023-01-08 06:46:20 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container c ready: false, restart count 0 Jan 8 06:51:00.024: INFO: ss-0 started at 2023-01-08 06:44:55 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container webserver ready: false, restart count 0 Jan 8 06:51:00.024: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-q97zd started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:00.024: INFO: ebs-csi-node-ft7bw started at 2023-01-08 06:25:38 +0000 UTC (0+3 container statuses recorded) Jan 8 06:51:00.024: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:51:00.024: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:51:00.024: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:51:00.024: INFO: pod-8fb38c56-a909-49a3-85f0-9d73b5275577 started at 2023-01-08 06:50:50 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container test-container ready: false, restart count 0 Jan 8 06:51:00.024: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2bn4h started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:00.024: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v9tjx started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:00.024: INFO: pod1 started at 2023-01-08 06:50:04 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container agnhost-container ready: false, restart count 0 Jan 8 06:51:00.024: INFO: service-proxy-disabled-6pp8x started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container service-proxy-disabled ready: true, restart count 0 Jan 8 06:51:00.024: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-97qp6 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:00.024: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-4d46c started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:00.024: INFO: inline-volume-tester2-lv7ch started at 2023-01-08 06:44:52 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:00.024: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 8 06:51:00.666: INFO: Latency metrics for node i-0685adc783a822be6 Jan 8 06:51:00.666: INFO: Logging node info for node i-0bd196cab651da520 Jan 8 06:51:00.843: INFO: Node Info: &Node{ObjectMeta:{i-0bd196cab651da520 f7f00780-d24e-4c8e-b54d-373598e1d1d2 41430 0 2023-01-08 06:26:02 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0bd196cab651da520 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0bd196cab651da520 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.104 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0bd196cab651da520"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-08 06:26:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:26:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:50:48 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-08 06:50:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0bd196cab651da520,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:50:54 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:50:54 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:50:54 +0000 UTC,LastTransitionTime:2023-01-08 06:26:02 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:50:54 +0000 UTC,LastTransitionTime:2023-01-08 06:26:16 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.104,},NodeAddress{Type:ExternalIP,Address:54.180.1.55,},NodeAddress{Type:InternalDNS,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0bd196cab651da520.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-180-1-55.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec218f9d32a84b2feeb6301255fd576c,SystemUUID:ec218f9d-32a8-4b2f-eeb6-301255fd576c,BootID:accdf7fd-1195-4765-b28e-29d44bf47bac,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-021c1df2b6aa5a7be kubernetes.io/csi/ebs.csi.aws.com^vol-02d83118eb07c1244 kubernetes.io/csi/ebs.csi.aws.com^vol-06c6e766891904b0f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-06c6e766891904b0f,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-021c1df2b6aa5a7be,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-02d83118eb07c1244,DevicePath:,},},Config:nil,},} Jan 8 06:51:00.844: INFO: Logging kubelet events for node i-0bd196cab651da520 Jan 8 06:51:01.023: INFO: Logging pods the kubelet thinks is on node i-0bd196cab651da520 Jan 8 06:51:01.209: INFO: hostexec-i-0bd196cab651da520-hvkk6 started at 2023-01-08 06:45:41 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container agnhost-container ready: true, restart count 0 Jan 8 06:51:01.209: INFO: pod-projected-configmaps-1b1db2bb-9759-4536-97eb-e0b261ef4c1b started at 2023-01-08 06:47:28 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container agnhost-container ready: false, restart count 0 Jan 8 06:51:01.209: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-6df8l started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:01.209: INFO: inline-volume-tester-hgbc6 started at 2023-01-08 06:50:45 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container csi-volume-tester ready: false, restart count 0 Jan 8 06:51:01.209: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-vrcjq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:01.209: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9776c started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:01.209: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-mphgx started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:01.209: INFO: oidc-discovery-validator started at 2023-01-08 06:51:00 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container oidc-discovery-validator ready: false, restart count 0 Jan 8 06:51:01.209: INFO: cilium-s6shx started at 2023-01-08 06:26:02 +0000 UTC (1+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:51:01.209: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:51:01.209: INFO: ebs-csi-node-6grlm started at 2023-01-08 06:26:02 +0000 UTC (0+3 container statuses recorded) Jan 8 06:51:01.209: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:51:01.209: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:51:01.209: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:51:01.209: INFO: webserver-pod started at 2023-01-08 06:46:47 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container agnhost ready: false, restart count 0 Jan 8 06:51:01.209: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9wskq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:01.209: INFO: service-headless-x4nfw started at 2023-01-08 06:50:25 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container service-headless ready: false, restart count 0 Jan 8 06:51:01.209: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-76nxw started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:01.209: INFO: pod-subpath-test-preprovisionedpv-brvp started at 2023-01-08 06:46:14 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container test-container-subpath-preprovisionedpv-brvp ready: false, restart count 0 Jan 8 06:51:01.209: INFO: hostexec-i-0bd196cab651da520-lk7nk started at 2023-01-08 06:50:41 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container agnhost-container ready: true, restart count 0 Jan 8 06:51:01.209: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-v44bk started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:01.209: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-75r8w started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:01.209: INFO: pod-a15c4150-c88e-49b1-a111-7f2271cf819c started at 2023-01-08 06:46:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container write-pod ready: false, restart count 0 Jan 8 06:51:01.209: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-cddx5 started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:01.209: INFO: netserver-1 started at 2023-01-08 06:46:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container webserver ready: true, restart count 0 Jan 8 06:51:01.209: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9mp69 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:01.209: INFO: pod-fea82a8a-61d5-4edb-b62d-f998d68a0314 started at 2023-01-08 06:50:46 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:01.209: INFO: Container write-pod ready: false, restart count 0 Jan 8 06:51:01.893: INFO: Latency metrics for node i-0bd196cab651da520 Jan 8 06:51:01.893: INFO: Logging node info for node i-0d149644e2edcf9cf Jan 8 06:51:02.070: INFO: Node Info: &Node{ObjectMeta:{i-0d149644e2edcf9cf b448ef72-6e61-4ad5-aa29-d63cc0f94945 41437 0 2023-01-08 06:25:37 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0d149644e2edcf9cf kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0d149644e2edcf9cf topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.35.115 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-8339":"i-0d149644e2edcf9cf","ebs.csi.aws.com":"i-0d149644e2edcf9cf"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-08 06:25:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-08 06:44:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-08 06:49:56 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-08 06:50:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0d149644e2edcf9cf,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051681280 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946823680 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-08 06:49:56 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-08 06:49:56 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-08 06:49:56 +0000 UTC,LastTransitionTime:2023-01-08 06:25:37 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-08 06:49:56 +0000 UTC,LastTransitionTime:2023-01-08 06:25:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.35.115,},NodeAddress{Type:ExternalIP,Address:3.35.219.218,},NodeAddress{Type:InternalDNS,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0d149644e2edcf9cf.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-35-219-218.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec26c6a9af66729e278545162595eb2d,SystemUUID:ec26c6a9-af66-729e-2785-45162595eb2d,BootID:959da9e3-fb86-442a-958f-2a49c509cdf4,KernelVersion:5.15.0-1026-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.14,KubeletVersion:v1.26.0,KubeProxyVersion:v1.26.0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.0],SizeBytes:67205320,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:10e231cf07c82dfab5141c1afe548dc734e5fc1f67665ec3982a325d1bd31a9a registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.0],SizeBytes:29725756,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-8339^1f8ae053-8f20-11ed-89ed-0a70e0670cb5],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-8339^1f8ae053-8f20-11ed-89ed-0a70e0670cb5,DevicePath:,},},Config:nil,},} Jan 8 06:51:02.070: INFO: Logging kubelet events for node i-0d149644e2edcf9cf Jan 8 06:51:02.250: INFO: Logging pods the kubelet thinks is on node i-0d149644e2edcf9cf Jan 8 06:51:02.434: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-rv5qq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:02.434: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-flhps started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:02.434: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-2vh6v started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0 Jan 8 06:51:02.434: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nt9jr started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0 Jan 8 06:51:02.434: INFO: cilium-58twl started at 2023-01-08 06:25:37 +0000 UTC (1+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 8 06:51:02.434: INFO: Container cilium-agent ready: true, restart count 0 Jan 8 06:51:02.434: INFO: coredns-559769c974-9q4qg started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Container coredns ready: true, restart count 0 Jan 8 06:51:02.434: INFO: csi-hostpathplugin-0 started at 2023-01-08 06:44:56 +0000 UTC (0+7 container statuses recorded) Jan 8 06:51:02.434: INFO: Container csi-attacher ready: true, restart count 0 Jan 8 06:51:02.434: INFO: Container csi-provisioner ready: true, restart count 0 Jan 8 06:51:02.434: INFO: Container csi-resizer ready: true, restart count 0 Jan 8 06:51:02.434: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 8 06:51:02.434: INFO: Container hostpath ready: true, restart count 0 Jan 8 06:51:02.434: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:51:02.434: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:51:02.434: INFO: ebs-csi-node-lfcwb started at 2023-01-08 06:25:37 +0000 UTC (0+3 container statuses recorded) Jan 8 06:51:02.434: INFO: Container ebs-plugin ready: true, restart count 0 Jan 8 06:51:02.434: INFO: Container liveness-probe ready: true, restart count 0 Jan 8 06:51:02.434: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 8 06:51:02.434: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-7gwsq started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:02.434: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-nsfnm started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:02.434: INFO: pod-b0cfeedb-9700-4fec-b239-25ba8a9fb152 started at 2023-01-08 06:46:07 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Container write-pod ready: false, restart count 0 Jan 8 06:51:02.434: INFO: netserver-2 started at 2023-01-08 06:46:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Container webserver ready: false, restart count 0 Jan 8 06:51:02.434: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-xrmq4 started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:02.434: INFO: service-proxy-disabled-v5f82 started at 2023-01-08 06:46:15 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Container service-proxy-disabled ready: true, restart count 0 Jan 8 06:51:02.434: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-tgn6r started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:02.434: INFO: coredns-autoscaler-7cb5c5b969-xqnk5 started at 2023-01-08 06:25:52 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Container autoscaler ready: true, restart count 0 Jan 8 06:51:02.434: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-9b2xq started at 2023-01-08 06:44:34 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: false, restart count 0 Jan 8 06:51:02.434: INFO: cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a-njh2f started at 2023-01-08 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 8 06:51:02.434: INFO: Container cleanup40-f787f707-b7b0-40e8-9b76-fcbc4a8fad9a ready: true, restart count 0 Jan 8 06:51:03.209: INFO: Latency metrics for node i-0d149644e2edcf9cf Jan 8 06:51:03.209: INFO: Logging node info for node i-0eee6113fd9f81d94 Jan 8 06:51:03.386: INFO: Node Info: &Node{ObjectMeta:{i-0eee6113fd9f81d94 6de592fc-f11c-4ba3-afa0-707bb4ca29b2 39949 0 2023-01-08 06:23:50 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0eee6113fd9f81d94 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.48.64 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0eee6113fd9f81d94"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-08 06:23:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-08 06:24:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-08 06:24:26 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-08 06:24:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-08 06:46:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransi