e2e-kops-grid-cilium-eni-u2004-k26 #1617769107304747008
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 32 failed / 860 succeeded |
| Started | |
| Elapsed | 47m46s |
| Revision | master |
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\sExternal\sStorage\s\[Driver\:\sebs\.csi\.aws\.com\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sfsgroupchangepolicy\s\(Always\)\[LinuxOnly\]\,\spod\screated\swith\san\sinitial\sfsgroup\,\svolume\scontents\sownership\schanged\svia\schgrp\sin\sfirst\spod\,\snew\spod\swith\ssame\sfsgroup\sapplied\sto\sthe\svolume\scontents$'
test/e2e/storage/testsuites/fsgroupchangepolicy.go:258
k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc0016f4f00, 0xc001525e10, 0x0, {0xc00217c4f0, 0x4}, {0xc00217c4f4, 0x4})
test/e2e/storage/testsuites/fsgroupchangepolicy.go:258 +0x151
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3()
test/e2e/storage/testsuites/fsgroupchangepolicy.go:247 +0x556
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/24/23 06:42:30.054�[0m
Jan 24 06:42:30.054: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename fsgroupchangepolicy �[38;5;243m01/24/23 06:42:30.055�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/24/23 06:42:30.543�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/24/23 06:42:30.866�[0m
[BeforeEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
test/e2e/framework/metrics/init/init.go:31
[It] (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with same fsgroup applied to the volume contents
test/e2e/storage/testsuites/fsgroupchangepolicy.go:210
Jan 24 06:42:31.188: INFO: Creating resource for dynamic PV
Jan 24 06:42:31.188: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi}
�[1mSTEP:�[0m creating a StorageClass fsgroupchangepolicy-4417-e2e-scdrfvm �[38;5;243m01/24/23 06:42:31.188�[0m
�[1mSTEP:�[0m creating a claim �[38;5;243m01/24/23 06:42:31.353�[0m
Jan 24 06:42:31.353: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
�[1mSTEP:�[0m Creating Pod in namespace fsgroupchangepolicy-4417 with fsgroup 1000 �[38;5;243m01/24/23 06:42:31.686�[0m
Jan 24 06:42:31.850: INFO: Waiting up to 5m0s for pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c" in namespace "fsgroupchangepolicy-4417" to be "running"
Jan 24 06:42:32.012: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c": Phase="Pending", Reason="", readiness=false. Elapsed: 161.519117ms
Jan 24 06:42:34.174: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c": Phase="Pending", Reason="", readiness=false. Elapsed: 2.323863452s
Jan 24 06:42:36.175: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c": Phase="Pending", Reason="", readiness=false. Elapsed: 4.325034994s
Jan 24 06:42:38.174: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c": Phase="Pending", Reason="", readiness=false. Elapsed: 6.323510364s
Jan 24 06:42:40.175: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c": Phase="Pending", Reason="", readiness=false. Elapsed: 8.324057478s
Jan 24 06:42:42.175: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c": Phase="Pending", Reason="", readiness=false. Elapsed: 10.324228913s
Jan 24 06:42:44.174: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c": Phase="Pending", Reason="", readiness=false. Elapsed: 12.323989541s
Jan 24 06:42:46.175: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c": Phase="Pending", Reason="", readiness=false. Elapsed: 14.324084385s
Jan 24 06:42:48.175: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c": Phase="Pending", Reason="", readiness=false. Elapsed: 16.32484351s
Jan 24 06:42:50.175: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c": Phase="Pending", Reason="", readiness=false. Elapsed: 18.324062915s
Jan 24 06:42:52.174: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c": Phase="Pending", Reason="", readiness=false. Elapsed: 20.323800629s
Jan 24 06:42:54.175: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c": Phase="Pending", Reason="", readiness=false. Elapsed: 22.324332847s
Jan 24 06:42:56.175: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c": Phase="Pending", Reason="", readiness=false. Elapsed: 24.324630122s
Jan 24 06:42:58.174: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c": Phase="Pending", Reason="", readiness=false. Elapsed: 26.323359524s
Jan 24 06:43:00.175: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c": Phase="Running", Reason="", readiness=true. Elapsed: 28.324151023s
Jan 24 06:43:00.175: INFO: Pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c" satisfied condition "running"
Jan 24 06:43:00.336: INFO: Pod fsgroupchangepolicy-4417/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c started successfully
�[1mSTEP:�[0m Creating a sub-directory and file, and verifying their ownership is 1000 �[38;5;243m01/24/23 06:43:00.336�[0m
Jan 24 06:43:00.337: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/file1] Namespace:fsgroupchangepolicy-4417 PodName:pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 24 06:43:00.337: INFO: >>> kubeConfig: /root/.kube/config
Jan 24 06:43:00.337: INFO: ExecWithOptions: Clientset creation
Jan 24 06:43:00.337: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-4417/pods/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c/exec?command=%2Fbin%2Fsh&command=-c&command=touch+%2Fmnt%2Fvolume1%2Ffile1&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 24 06:43:01.424: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/file1] Namespace:fsgroupchangepolicy-4417 PodName:pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 24 06:43:01.424: INFO: >>> kubeConfig: /root/.kube/config
Jan 24 06:43:01.424: INFO: ExecWithOptions: Clientset creation
Jan 24 06:43:01.424: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-4417/pods/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c/exec?command=%2Fbin%2Fsh&command=-c&command=ls+-l+%2Fmnt%2Fvolume1%2Ffile1&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 24 06:43:02.460: INFO: pod fsgroupchangepolicy-4417/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c exec for cmd ls -l /mnt/volume1/file1, stdout: -rw-r--r-- 1 root 1000 0 Jan 24 06:43 /mnt/volume1/file1, stderr:
Jan 24 06:43:02.460: INFO: stdout split: [-rw-r--r-- 1 root 1000 0 Jan 24 06:43 /mnt/volume1/file1], expected gid: 1000
Jan 24 06:43:02.460: INFO: ExecWithOptions {Command:[/bin/sh -c mkdir /mnt/volume1/subdir] Namespace:fsgroupchangepolicy-4417 PodName:pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 24 06:43:02.460: INFO: >>> kubeConfig: /root/.kube/config
Jan 24 06:43:02.461: INFO: ExecWithOptions: Clientset creation
Jan 24 06:43:02.461: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-4417/pods/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c/exec?command=%2Fbin%2Fsh&command=-c&command=mkdir+%2Fmnt%2Fvolume1%2Fsubdir&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 24 06:43:03.527: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-4417 PodName:pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 24 06:43:03.527: INFO: >>> kubeConfig: /root/.kube/config
Jan 24 06:43:03.528: INFO: ExecWithOptions: Clientset creation
Jan 24 06:43:03.528: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-4417/pods/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c/exec?command=%2Fbin%2Fsh&command=-c&command=touch+%2Fmnt%2Fvolume1%2Fsubdir%2Ffile2&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 24 06:43:04.589: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-4417 PodName:pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 24 06:43:04.589: INFO: >>> kubeConfig: /root/.kube/config
Jan 24 06:43:04.590: INFO: ExecWithOptions: Clientset creation
Jan 24 06:43:04.590: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-4417/pods/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c/exec?command=%2Fbin%2Fsh&command=-c&command=ls+-l+%2Fmnt%2Fvolume1%2Fsubdir%2Ffile2&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 24 06:43:05.626: INFO: pod fsgroupchangepolicy-4417/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r-- 1 root 1000 0 Jan 24 06:43 /mnt/volume1/subdir/file2, stderr:
Jan 24 06:43:05.626: INFO: stdout split: [-rw-r--r-- 1 root 1000 0 Jan 24 06:43 /mnt/volume1/subdir/file2], expected gid: 1000
�[1mSTEP:�[0m Changing the root directory file ownership to 2000 �[38;5;243m01/24/23 06:43:05.626�[0m
Jan 24 06:43:05.626: INFO: ExecWithOptions {Command:[/bin/sh -c chgrp 2000 /mnt/volume1/file1] Namespace:fsgroupchangepolicy-4417 PodName:pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 24 06:43:05.626: INFO: >>> kubeConfig: /root/.kube/config
Jan 24 06:43:05.627: INFO: ExecWithOptions: Clientset creation
Jan 24 06:43:05.627: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-4417/pods/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c/exec?command=%2Fbin%2Fsh&command=-c&command=chgrp+2000+%2Fmnt%2Fvolume1%2Ffile1&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 24 06:43:06.684: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/file1] Namespace:fsgroupchangepolicy-4417 PodName:pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 24 06:43:06.684: INFO: >>> kubeConfig: /root/.kube/config
Jan 24 06:43:06.685: INFO: ExecWithOptions: Clientset creation
Jan 24 06:43:06.685: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-4417/pods/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c/exec?command=%2Fbin%2Fsh&command=-c&command=ls+-l+%2Fmnt%2Fvolume1%2Ffile1&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 24 06:43:07.754: INFO: pod fsgroupchangepolicy-4417/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c exec for cmd ls -l /mnt/volume1/file1, stdout: -rw-r--r-- 1 root 2000 0 Jan 24 06:43 /mnt/volume1/file1, stderr:
Jan 24 06:43:07.754: INFO: stdout split: [-rw-r--r-- 1 root 2000 0 Jan 24 06:43 /mnt/volume1/file1], expected gid: 2000
�[1mSTEP:�[0m Changing the sub-directory file ownership to 3000 �[38;5;243m01/24/23 06:43:07.754�[0m
Jan 24 06:43:07.754: INFO: ExecWithOptions {Command:[/bin/sh -c chgrp 3000 /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-4417 PodName:pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 24 06:43:07.754: INFO: >>> kubeConfig: /root/.kube/config
Jan 24 06:43:07.755: INFO: ExecWithOptions: Clientset creation
Jan 24 06:43:07.755: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-4417/pods/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c/exec?command=%2Fbin%2Fsh&command=-c&command=chgrp+3000+%2Fmnt%2Fvolume1%2Fsubdir%2Ffile2&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 24 06:43:08.824: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-4417 PodName:pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Jan 24 06:43:08.824: INFO: >>> kubeConfig: /root/.kube/config
Jan 24 06:43:08.825: INFO: ExecWithOptions: Clientset creation
Jan 24 06:43:08.825: INFO: ExecWithOptions: execute(POST https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io/api/v1/namespaces/fsgroupchangepolicy-4417/pods/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c/exec?command=%2Fbin%2Fsh&command=-c&command=ls+-l+%2Fmnt%2Fvolume1%2Fsubdir%2Ffile2&container=write-pod&container=write-pod&stderr=true&stdout=true)
Jan 24 06:43:09.860: INFO: pod fsgroupchangepolicy-4417/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r-- 1 root 3000 0 Jan 24 06:43 /mnt/volume1/subdir/file2, stderr:
Jan 24 06:43:09.860: INFO: stdout split: [-rw-r--r-- 1 root 3000 0 Jan 24 06:43 /mnt/volume1/subdir/file2], expected gid: 3000
�[1mSTEP:�[0m Deleting Pod fsgroupchangepolicy-4417/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c �[38;5;243m01/24/23 06:43:09.86�[0m
Jan 24 06:43:09.860: INFO: Deleting pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c" in namespace "fsgroupchangepolicy-4417"
Jan 24 06:43:10.026: INFO: Wait up to 5m0s for pod "pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c" to be fully deleted
�[1mSTEP:�[0m Creating Pod in namespace fsgroupchangepolicy-4417 with fsgroup 1000 �[38;5;243m01/24/23 06:43:12.35�[0m
Jan 24 06:43:12.517: INFO: Waiting up to 5m0s for pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911" in namespace "fsgroupchangepolicy-4417" to be "running"
Jan 24 06:43:12.678: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 161.453813ms
Jan 24 06:43:14.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2.324178054s
Jan 24 06:43:16.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4.324139638s
Jan 24 06:43:18.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 6.323971126s
Jan 24 06:43:20.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 8.323976657s
Jan 24 06:43:22.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 10.324264197s
Jan 24 06:43:24.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 12.324486057s
Jan 24 06:43:26.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 14.32388483s
Jan 24 06:43:28.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 16.323729539s
Jan 24 06:43:30.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 18.32369533s
Jan 24 06:43:32.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 20.324247994s
Jan 24 06:43:34.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 22.324192525s
Jan 24 06:43:36.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 24.324056719s
Jan 24 06:43:38.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 26.324069254s
Jan 24 06:43:40.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 28.323977046s
Jan 24 06:43:42.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 30.32367245s
Jan 24 06:43:44.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 32.323621047s
Jan 24 06:43:46.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 34.323546451s
Jan 24 06:43:48.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 36.323837329s
Jan 24 06:43:50.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 38.32446047s
Jan 24 06:43:52.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 40.323896447s
Jan 24 06:43:54.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 42.323561397s
Jan 24 06:43:56.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 44.324315831s
Jan 24 06:43:58.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 46.324198891s
Jan 24 06:44:00.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 48.323528082s
Jan 24 06:44:02.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 50.323587557s
Jan 24 06:44:04.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 52.32421254s
Jan 24 06:44:06.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 54.323823356s
Jan 24 06:44:08.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 56.323390971s
Jan 24 06:44:10.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 58.324013013s
Jan 24 06:44:12.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.323872461s
Jan 24 06:44:14.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.324040946s
Jan 24 06:44:16.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.324073601s
Jan 24 06:44:18.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.323470794s
Jan 24 06:44:20.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.324357482s
Jan 24 06:44:22.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.324116076s
Jan 24 06:44:24.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.324184487s
Jan 24 06:44:26.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.323927241s
Jan 24 06:44:28.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.323765059s
Jan 24 06:44:30.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.32373401s
Jan 24 06:44:32.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.323965049s
Jan 24 06:44:34.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.323573427s
Jan 24 06:44:36.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.323529741s
Jan 24 06:44:38.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.323898477s
Jan 24 06:44:40.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.324251268s
Jan 24 06:44:42.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.324251722s
Jan 24 06:44:44.843: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.32614337s
Jan 24 06:44:46.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.324418848s
Jan 24 06:44:48.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.323440215s
Jan 24 06:44:50.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.323534263s
Jan 24 06:44:52.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.324083889s
Jan 24 06:44:54.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.3241073s
Jan 24 06:44:56.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.323546691s
Jan 24 06:44:58.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.323821419s
Jan 24 06:45:00.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.32346112s
Jan 24 06:45:02.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.324167736s
Jan 24 06:45:04.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.323446129s
Jan 24 06:45:06.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.323973514s
Jan 24 06:45:08.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.323517242s
Jan 24 06:45:10.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.323959436s
Jan 24 06:45:12.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.324017805s
Jan 24 06:45:14.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.323806671s
Jan 24 06:45:16.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.323685515s
Jan 24 06:45:18.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.324318863s
Jan 24 06:45:20.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.324015892s
Jan 24 06:45:22.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.323474093s
Jan 24 06:45:24.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.324052857s
Jan 24 06:45:26.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.32382126s
Jan 24 06:45:28.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.323341077s
Jan 24 06:45:30.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.323878384s
Jan 24 06:45:32.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.323336151s
Jan 24 06:45:34.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.323764965s
Jan 24 06:45:36.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.323146523s
Jan 24 06:45:38.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.323690855s
Jan 24 06:45:40.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.324255969s
Jan 24 06:45:42.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.323704523s
Jan 24 06:45:44.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.323508275s
Jan 24 06:45:46.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.324075513s
Jan 24 06:45:48.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.3242638s
Jan 24 06:45:50.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.323937666s
Jan 24 06:45:52.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.324149412s
Jan 24 06:45:54.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.32340254s
Jan 24 06:45:56.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.324375246s
Jan 24 06:45:58.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.323559189s
Jan 24 06:46:00.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.323716988s
Jan 24 06:46:02.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.323836896s
Jan 24 06:46:04.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.323677805s
Jan 24 06:46:06.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.323822844s
Jan 24 06:46:08.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.323618505s
Jan 24 06:46:10.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.323784583s
Jan 24 06:46:12.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.323652944s
Jan 24 06:46:14.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.323808977s
Jan 24 06:46:16.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.323735243s
Jan 24 06:46:18.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.323891059s
Jan 24 06:46:20.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.324042849s
Jan 24 06:46:22.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.323790883s
Jan 24 06:46:24.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.323580894s
Jan 24 06:46:26.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.323347369s
Jan 24 06:46:28.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.323444214s
Jan 24 06:46:30.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.324110599s
Jan 24 06:46:32.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.323727093s
Jan 24 06:46:34.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.324005692s
Jan 24 06:46:36.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.323499333s
Jan 24 06:46:38.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.323988459s
Jan 24 06:46:40.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.324162191s
Jan 24 06:46:42.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.324149482s
Jan 24 06:46:44.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.324364872s
Jan 24 06:46:46.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.323517513s
Jan 24 06:46:48.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.324324082s
Jan 24 06:46:50.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.323539787s
Jan 24 06:46:52.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.324476806s
Jan 24 06:46:54.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.323811882s
Jan 24 06:46:56.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.323357748s
Jan 24 06:46:58.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.323637765s
Jan 24 06:47:00.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.323973866s
Jan 24 06:47:02.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.324594712s
Jan 24 06:47:04.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.323607083s
Jan 24 06:47:06.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.323936126s
Jan 24 06:47:08.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.324053415s
Jan 24 06:47:10.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.324160753s
Jan 24 06:47:12.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.323162108s
Jan 24 06:47:14.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.324329181s
Jan 24 06:47:16.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.323982919s
Jan 24 06:47:18.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.323907918s
Jan 24 06:47:20.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.323701437s
Jan 24 06:47:22.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.324398606s
Jan 24 06:47:24.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.323185442s
Jan 24 06:47:26.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.323780871s
Jan 24 06:47:28.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.323772551s
Jan 24 06:47:30.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.323731199s
Jan 24 06:47:32.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.32341347s
Jan 24 06:47:34.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.323969662s
Jan 24 06:47:36.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.323261864s
Jan 24 06:47:38.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.323843853s
Jan 24 06:47:40.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.324091447s
Jan 24 06:47:42.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.323884678s
Jan 24 06:47:44.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.323701074s
Jan 24 06:47:46.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.323769617s
Jan 24 06:47:48.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.323878774s
Jan 24 06:47:50.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.323866489s
Jan 24 06:47:52.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.32373513s
Jan 24 06:47:54.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.323507043s
Jan 24 06:47:56.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.323463036s
Jan 24 06:47:58.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.324003076s
Jan 24 06:48:00.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.323259064s
Jan 24 06:48:02.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.323202938s
Jan 24 06:48:04.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.323619062s
Jan 24 06:48:06.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.324073087s
Jan 24 06:48:08.840: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.323472883s
Jan 24 06:48:10.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.323939776s
Jan 24 06:48:12.841: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.324180886s
Jan 24 06:48:13.003: INFO: Pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.485897877s
Jan 24 06:48:13.003: INFO: Unexpected error:
<*errors.errorString | 0xc0010b3a90>: {
s: "pod \"pod-da9584f9-0d79-4167-ad37-ca206c3ac911\" is not Running: timed out while waiting for pod fsgroupchangepolicy-4417/pod-da9584f9-0d79-4167-ad37-ca206c3ac911 to be running",
}
Jan 24 06:48:13.003: FAIL: pod "pod-da9584f9-0d79-4167-ad37-ca206c3ac911" is not Running: timed out while waiting for pod fsgroupchangepolicy-4417/pod-da9584f9-0d79-4167-ad37-ca206c3ac911 to be running
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc0016f4f00, 0xc001525e10, 0x0, {0xc00217c4f0, 0x4}, {0xc00217c4f4, 0x4})
test/e2e/storage/testsuites/fsgroupchangepolicy.go:258 +0x151
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3()
test/e2e/storage/testsuites/fsgroupchangepolicy.go:247 +0x556
�[1mSTEP:�[0m Deleting pvc �[38;5;243m01/24/23 06:48:13.003�[0m
Jan 24 06:48:13.326: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com6vr78"
Jan 24 06:48:13.489: INFO: Waiting up to 5m0s for PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 to get deleted
Jan 24 06:48:13.651: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (161.547239ms)
Jan 24 06:48:18.813: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (5.324037501s)
Jan 24 06:48:23.976: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (10.486796079s)
Jan 24 06:48:29.138: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (15.648668681s)
Jan 24 06:48:34.302: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (20.812418294s)
Jan 24 06:48:39.465: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (25.975335411s)
Jan 24 06:48:44.628: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (31.138761691s)
Jan 24 06:48:49.790: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (36.300782323s)
Jan 24 06:48:54.959: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (41.469953865s)
Jan 24 06:49:00.121: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (46.632286525s)
Jan 24 06:49:05.283: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (51.793766477s)
Jan 24 06:49:10.445: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (56.955669443s)
Jan 24 06:49:15.607: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (1m2.117661949s)
Jan 24 06:49:20.771: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (1m7.281390182s)
Jan 24 06:49:25.933: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (1m12.44375954s)
Jan 24 06:49:31.096: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (1m17.606373317s)
Jan 24 06:49:36.257: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (1m22.768134203s)
Jan 24 06:49:41.419: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (1m27.930170654s)
Jan 24 06:49:46.582: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (1m33.092826232s)
Jan 24 06:49:51.744: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (1m38.255074308s)
Jan 24 06:49:56.907: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (1m43.417985204s)
Jan 24 06:50:02.070: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (1m48.580694921s)
Jan 24 06:50:07.232: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (1m53.742726404s)
Jan 24 06:50:12.394: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (1m58.904714134s)
Jan 24 06:50:17.557: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (2m4.067776733s)
Jan 24 06:50:22.720: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (2m9.230712179s)
Jan 24 06:50:27.883: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (2m14.393367546s)
Jan 24 06:50:33.045: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (2m19.555365717s)
Jan 24 06:50:38.207: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (2m24.717450862s)
Jan 24 06:50:43.369: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (2m29.88003989s)
Jan 24 06:50:48.532: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (2m35.042712409s)
Jan 24 06:50:53.694: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (2m40.205207513s)
Jan 24 06:50:58.856: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (2m45.367283108s)
Jan 24 06:51:04.019: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (2m50.530098951s)
Jan 24 06:51:09.182: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (2m55.692775115s)
Jan 24 06:51:14.345: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (3m0.855339738s)
Jan 24 06:51:19.507: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (3m6.017602551s)
Jan 24 06:51:24.669: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (3m11.179355951s)
Jan 24 06:51:29.831: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (3m16.342241329s)
Jan 24 06:51:34.994: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (3m21.504833469s)
Jan 24 06:51:40.157: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (3m26.667600006s)
Jan 24 06:51:45.319: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (3m31.830156589s)
Jan 24 06:51:50.481: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (3m36.991933861s)
Jan 24 06:51:55.644: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (3m42.154638167s)
Jan 24 06:52:00.806: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (3m47.316899597s)
Jan 24 06:52:05.969: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (3m52.479402725s)
Jan 24 06:52:11.131: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (3m57.641420168s)
Jan 24 06:52:16.293: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (4m2.804074631s)
Jan 24 06:52:21.456: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (4m7.966628592s)
Jan 24 06:52:26.619: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (4m13.129656141s)
Jan 24 06:52:31.782: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (4m18.292450995s)
Jan 24 06:52:36.943: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (4m23.453959629s)
Jan 24 06:52:42.105: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (4m28.616283777s)
Jan 24 06:52:47.268: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (4m33.778799054s)
Jan 24 06:52:52.431: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (4m38.941497654s)
Jan 24 06:52:57.593: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (4m44.103464728s)
Jan 24 06:53:02.754: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (4m49.265277923s)
Jan 24 06:53:07.917: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (4m54.427808723s)
Jan 24 06:53:13.080: INFO: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 found and phase=Bound (4m59.590360677s)
�[1mSTEP:�[0m Deleting sc �[38;5;243m01/24/23 06:53:18.08�[0m
Jan 24 06:53:18.243: INFO: Unexpected error: while cleanup resource:
<errors.aggregate | len:1, cap:1>: [
<errors.aggregate | len:1, cap:1>[
<*fmt.wrapError | 0xc000c00200>{
msg: "persistent Volume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 not deleted by dynamic provisioner: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 still exists within 5m0s",
err: <*errors.errorString | 0xc001458590>{
s: "PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 still exists within 5m0s",
},
},
],
]
Jan 24 06:53:18.243: FAIL: while cleanup resource: persistent Volume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 not deleted by dynamic provisioner: PersistentVolume pvc-1adb5c95-d901-457b-99a4-a87d68b79950 still exists within 5m0s
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2()
test/e2e/storage/testsuites/fsgroupchangepolicy.go:128 +0x14e
panic({0x70efe60, 0xc00062d420})
/usr/local/go/src/runtime/panic.go:884 +0x212
k8s.io/kubernetes/test/e2e/framework.Fail({0xc002cf0000, 0xae}, {0xc001fabc30?, 0xc002cf0000?, 0xc001fabc58?})
test/e2e/framework/log.go:61 +0x145
k8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0x1, {0x7fa8f20, 0xc0010b3a90}, {0x0?, 0x2?, 0x0?})
test/e2e/framework/expect.go:76 +0x267
k8s.io/kubernetes/test/e2e/framework.ExpectNoError(...)
test/e2e/framework/expect.go:43
k8s.io/kubernetes/test/e2e/storage/testsuites.createPodAndVerifyContentGid(0xc0016f4f00, 0xc001525e10, 0x0, {0xc00217c4f0, 0x4}, {0xc00217c4f4, 0x4})
test/e2e/storage/testsuites/fsgroupchangepolicy.go:258 +0x151
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func3()
test/e2e/storage/testsuites/fsgroupchangepolicy.go:247 +0x556
[AfterEach] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
test/e2e/framework/node/init/init.go:32
Jan 24 06:53:18.244: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/24/23 06:53:18.407�[0m
�[1mSTEP:�[0m Collecting events from namespace "fsgroupchangepolicy-4417". �[38;5;243m01/24/23 06:53:18.407�[0m
�[1mSTEP:�[0m Found 22 events. �[38;5;243m01/24/23 06:53:18.569�[0m
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:42:31 +0000 UTC - event for ebs.csi.aws.com6vr78: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:42:31 +0000 UTC - event for ebs.csi.aws.com6vr78: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:42:31 +0000 UTC - event for ebs.csi.aws.com6vr78: {ebs.csi.aws.com_i-0b0b3ac061164cea4_77bfeef8-656d-4806-b8d4-c989baee5392 } Provisioning: External provisioner is provisioning volume for claim "fsgroupchangepolicy-4417/ebs.csi.aws.com6vr78"
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:42:35 +0000 UTC - event for ebs.csi.aws.com6vr78: {ebs.csi.aws.com_i-0b0b3ac061164cea4_77bfeef8-656d-4806-b8d4-c989baee5392 } ProvisioningSucceeded: Successfully provisioned volume pvc-1adb5c95-d901-457b-99a4-a87d68b79950
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:42:35 +0000 UTC - event for pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c: {default-scheduler } Scheduled: Successfully assigned fsgroupchangepolicy-4417/pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c to i-036f641c68d5ad392
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:42:51 +0000 UTC - event for pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-1adb5c95-d901-457b-99a4-a87d68b79950" : rpc error: code = Internal desc = Could not attach volume "vol-0269089ae9a4173b7" to node "i-036f641c68d5ad392": attachment of disk "vol-0269089ae9a4173b7" failed, expected device to be attached but was attaching
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:42:57 +0000 UTC - event for pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-1adb5c95-d901-457b-99a4-a87d68b79950"
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:42:58 +0000 UTC - event for pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c: {kubelet i-036f641c68d5ad392} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:42:58 +0000 UTC - event for pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c: {kubelet i-036f641c68d5ad392} Created: Created container write-pod
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:42:58 +0000 UTC - event for pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c: {kubelet i-036f641c68d5ad392} Started: Started container write-pod
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:43:09 +0000 UTC - event for pod-5482c295-8cd8-4f7f-bc1e-29919ed0b75c: {kubelet i-036f641c68d5ad392} Killing: Stopping container write-pod
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:43:12 +0000 UTC - event for pod-da9584f9-0d79-4167-ad37-ca206c3ac911: {default-scheduler } Scheduled: Successfully assigned fsgroupchangepolicy-4417/pod-da9584f9-0d79-4167-ad37-ca206c3ac911 to i-036f641c68d5ad392
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:43:17 +0000 UTC - event for pod-da9584f9-0d79-4167-ad37-ca206c3ac911: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d1c96f32047adc13ccb2e2ac0495205ccc77d66bd309c5ae86d730fcc9dea463": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:43:31 +0000 UTC - event for pod-da9584f9-0d79-4167-ad37-ca206c3ac911: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e7e0e76c3e803e47669c16152d58b87cc38ab164f6267eba15f10ffdc1c52bdf": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:43:44 +0000 UTC - event for pod-da9584f9-0d79-4167-ad37-ca206c3ac911: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6c6932a9d6972873b01638018d87d9cc58ebe427ca4e2c7fa7b9d5c187794926": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:43:59 +0000 UTC - event for pod-da9584f9-0d79-4167-ad37-ca206c3ac911: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e45d5943c37c4e1664a28d0402009a3cc5200644787bf86029d658b37675520c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:44:10 +0000 UTC - event for pod-da9584f9-0d79-4167-ad37-ca206c3ac911: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "162ea92afd00b1b4ae97a9f1f6e3e5cb61ad1e10211f36bd94d99c069021f1d9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:44:22 +0000 UTC - event for pod-da9584f9-0d79-4167-ad37-ca206c3ac911: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "88e5e309f2b661f20f479c8d45c6757e2b32bada071e9566373cd66d2f4ff0dd": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:44:34 +0000 UTC - event for pod-da9584f9-0d79-4167-ad37-ca206c3ac911: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cf9d72cf995dc33c7f7c12841d3333d97c92b3cd7bc95048290f8b7dc66da599": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:44:46 +0000 UTC - event for pod-da9584f9-0d79-4167-ad37-ca206c3ac911: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6ff52366fff8c007469ac8e4466e1694b5bc737be5dee9f5485bb6606f35b6f4": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:44:57 +0000 UTC - event for pod-da9584f9-0d79-4167-ad37-ca206c3ac911: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "02da78c707240f4da3883af4927a37a6b981eb8093572f14685128d0ddb341b9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:18.569: INFO: At 2023-01-24 06:45:08 +0000 UTC - event for pod-da9584f9-0d79-4167-ad37-ca206c3ac911: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c39af383eae2beced0cbe72b40462cae3d293f2e670420271bb48038445069fb": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:18.731: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 24 06:53:18.731: INFO: pod-da9584f9-0d79-4167-ad37-ca206c3ac911 i-036f641c68d5ad392 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:12 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:12 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:12 +0000 UTC ContainersNotReady containers with unready status: [write-pod]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:12 +0000 UTC }]
Jan 24 06:53:18.731: INFO:
Jan 24 06:53:18.894: INFO: Unable to fetch fsgroupchangepolicy-4417/pod-da9584f9-0d79-4167-ad37-ca206c3ac911/write-pod logs: the server rejected our request for an unknown reason (get pods pod-da9584f9-0d79-4167-ad37-ca206c3ac911)
Jan 24 06:53:19.057: INFO:
Logging node info for node i-036f641c68d5ad392
Jan 24 06:53:19.219: INFO: Node Info: &Node{ObjectMeta:{i-036f641c68d5ad392 50721840-3696-4207-81a9-8cd089fbb539 33635 0 2023-01-24 06:27:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-036f641c68d5ad392 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-036f641c68d5ad392 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.254 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-5409":"i-036f641c68d5ad392","ebs.csi.aws.com":"i-036f641c68d5ad392"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:48:57 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-24 06:49:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-036f641c68d5ad392,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:04 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:04 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:04 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:49:04 +0000 UTC,LastTransitionTime:2023-01-24 06:27:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.254,},NodeAddress{Type:ExternalIP,Address:15.165.237.223,},NodeAddress{Type:InternalDNS,Address:i-036f641c68d5ad392.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-036f641c68d5ad392.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-165-237-223.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2732c784a9bb7e64a787956c82cf85,SystemUUID:ec2732c7-84a9-bb7e-64a7-87956c82cf85,BootID:bee5ed40-42b1-475e-b72b-16d715de535a,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-5409^2c26217c-9bb3-11ed-bce5-86d6f2bc551f kubernetes.io/csi/ebs.csi.aws.com^vol-0269089ae9a4173b7],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0269089ae9a4173b7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-5409^2c26217c-9bb3-11ed-bce5-86d6f2bc551f,DevicePath:,},},Config:nil,},}
Jan 24 06:53:19.219: INFO:
Logging kubelet events for node i-036f641c68d5ad392
Jan 24 06:53:19.425: INFO:
Logging pods the kubelet thinks is on node i-036f641c68d5ad392
Jan 24 06:53:19.595: INFO: rs-76gck started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.595: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:19.596: INFO: pfpod started at 2023-01-24 06:48:43 +0000 UTC (0+2 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container portforwardtester ready: false, restart count 0
Jan 24 06:53:19.596: INFO: Container readiness ready: false, restart count 0
Jan 24 06:53:19.596: INFO: dns-test-93a0bf56-7463-4d4f-a334-bb3e44707aaa started at 2023-01-24 06:51:48 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container jessie-querier ready: false, restart count 0
Jan 24 06:53:19.596: INFO: Container querier ready: false, restart count 0
Jan 24 06:53:19.596: INFO: Container webserver ready: false, restart count 0
Jan 24 06:53:19.596: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6twph started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:19.596: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-m4sr7 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:19.596: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2mkph started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:19.596: INFO: rs-6s4df started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:19.596: INFO: csi-hostpathplugin-0 started at 2023-01-24 06:44:58 +0000 UTC (0+7 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container csi-attacher ready: true, restart count 0
Jan 24 06:53:19.596: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:53:19.596: INFO: Container csi-resizer ready: true, restart count 0
Jan 24 06:53:19.596: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 24 06:53:19.596: INFO: Container hostpath ready: true, restart count 0
Jan 24 06:53:19.596: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:19.596: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:19.596: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-8sfzb started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:19.596: INFO: rs-zmrj2 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:19.596: INFO: pod-projected-configmaps-495807fd-45a6-4d3f-a257-d364553fb370 started at 2023-01-24 06:48:40 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container agnhost-container ready: false, restart count 0
Jan 24 06:53:19.596: INFO: ebs-csi-node-6z65r started at 2023-01-24 06:27:40 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:19.596: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:19.596: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:19.596: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cpp7h started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:19.596: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6hnbl started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:19.596: INFO: service-headless-btk74 started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container service-headless ready: false, restart count 0
Jan 24 06:53:19.596: INFO: pod-da9584f9-0d79-4167-ad37-ca206c3ac911 started at 2023-01-24 06:43:12 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container write-pod ready: false, restart count 0
Jan 24 06:53:19.596: INFO: pod-handle-http-request started at 2023-01-24 06:48:15 +0000 UTC (0+2 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container container-handle-http-request ready: false, restart count 0
Jan 24 06:53:19.596: INFO: Container container-handle-https-request ready: false, restart count 0
Jan 24 06:53:19.596: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-h5mpm started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:19.596: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-tcp26 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:19.596: INFO: cilium-9ht2g started at 2023-01-24 06:27:40 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:53:19.596: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:53:19.596: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-t9b6f started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:19.596: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-lc7s4 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:19.596: INFO: rand-non-local-gxd9p started at 2023-01-24 06:42:31 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container c ready: false, restart count 0
Jan 24 06:53:19.596: INFO: pod-56c7bad9-2975-4685-aab0-0cfb8b42331c started at 2023-01-24 06:48:56 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container write-pod ready: false, restart count 0
Jan 24 06:53:19.596: INFO: server started at 2023-01-24 06:49:10 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container agnhost-container ready: true, restart count 0
Jan 24 06:53:19.596: INFO: test-ss-0 started at 2023-01-24 06:49:23 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container webserver ready: false, restart count 0
Jan 24 06:53:19.596: INFO: pod-configmaps-5f55ab46-5e11-4c96-87a9-f170ab7204ce started at 2023-01-24 06:49:28 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:19.596: INFO: Container agnhost-container ready: false, restart count 0
Jan 24 06:53:20.255: INFO:
Latency metrics for node i-036f641c68d5ad392
Jan 24 06:53:20.255: INFO:
Logging node info for node i-0697a70b9aa95b7c0
Jan 24 06:53:20.417: INFO: Node Info: &Node{ObjectMeta:{i-0697a70b9aa95b7c0 d0a3c976-eb07-4323-b48b-ec4945de7410 33229 0 2023-01-24 06:27:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0697a70b9aa95b7c0 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0697a70b9aa95b7c0 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.153 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0697a70b9aa95b7c0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:48:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0697a70b9aa95b7c0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:49 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.153,},NodeAddress{Type:ExternalIP,Address:3.38.96.49,},NodeAddress{Type:InternalDNS,Address:i-0697a70b9aa95b7c0.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0697a70b9aa95b7c0.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-38-96-49.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f1e14d470f251c9d042c57478cb8b,SystemUUID:ec2f1e14-d470-f251-c9d0-42c57478cb8b,BootID:578a4c1d-084e-4aeb-92fe-f64671842d82,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:53:20.418: INFO:
Logging kubelet events for node i-0697a70b9aa95b7c0
Jan 24 06:53:20.582: INFO:
Logging pods the kubelet thinks is on node i-0697a70b9aa95b7c0
Jan 24 06:53:20.751: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-68w92 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:20.751: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qphds started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:20.751: INFO: service-headless-xh4pl started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container service-headless ready: false, restart count 0
Jan 24 06:53:20.751: INFO: csi-mockplugin-0 started at 2023-01-24 06:48:54 +0000 UTC (0+4 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container busybox ready: false, restart count 0
Jan 24 06:53:20.751: INFO: Container csi-provisioner ready: false, restart count 0
Jan 24 06:53:20.751: INFO: Container driver-registrar ready: false, restart count 0
Jan 24 06:53:20.751: INFO: Container mock ready: false, restart count 0
Jan 24 06:53:20.751: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qtzq5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:20.751: INFO: rs-b9vhw started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container donothing ready: true, restart count 0
Jan 24 06:53:20.751: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2tkvs started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:53:20.751: INFO: coredns-559769c974-6wblg started at 2023-01-24 06:27:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container coredns ready: true, restart count 0
Jan 24 06:53:20.751: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-5pcw5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:20.751: INFO: cilium-dh7hm started at 2023-01-24 06:27:36 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:53:20.751: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:53:20.751: INFO: rs-w9wjl started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container donothing ready: true, restart count 0
Jan 24 06:53:20.751: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cdkdr started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:20.751: INFO: pod-subpath-test-preprovisionedpv-94gx started at 2023-01-24 06:50:18 +0000 UTC (2+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Init container init-volume-preprovisionedpv-94gx ready: false, restart count 0
Jan 24 06:53:20.751: INFO: Init container test-init-volume-preprovisionedpv-94gx ready: false, restart count 0
Jan 24 06:53:20.751: INFO: Container test-container-subpath-preprovisionedpv-94gx ready: false, restart count 0
Jan 24 06:53:20.751: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-xbpd8 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:53:20.751: INFO: coredns-autoscaler-7cb5c5b969-vktlf started at 2023-01-24 06:27:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container autoscaler ready: true, restart count 0
Jan 24 06:53:20.751: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-fgc56 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:20.751: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2v7c5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:20.751: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-4sk4h started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:20.751: INFO: hostexec-i-0697a70b9aa95b7c0-n9ccb started at 2023-01-24 06:49:56 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container agnhost-container ready: true, restart count 0
Jan 24 06:53:20.751: INFO: ebs-csi-node-pkrr4 started at 2023-01-24 06:27:36 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:20.751: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:20.751: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:20.751: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:21.368: INFO:
Latency metrics for node i-0697a70b9aa95b7c0
Jan 24 06:53:21.368: INFO:
Logging node info for node i-0b0b3ac061164cea4
Jan 24 06:53:21.530: INFO: Node Info: &Node{ObjectMeta:{i-0b0b3ac061164cea4 ac00aaf7-4427-49f6-b849-9c812ae80069 34896 0 2023-01-24 06:25:23 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b0b3ac061164cea4 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.51.48 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b0b3ac061164cea4"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-24 06:25:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-24 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:26:22 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:26:22 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-24 06:26:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-24 06:53:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0b0b3ac061164cea4,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892297728 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787440128 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:53:16 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:53:16 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:53:16 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:53:16 +0000 UTC,LastTransitionTime:2023-01-24 06:26:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.48,},NodeAddress{Type:ExternalIP,Address:13.125.222.159,},NodeAddress{Type:InternalDNS,Address:i-0b0b3ac061164cea4.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b0b3ac061164cea4.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-125-222-159.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255807a7dec89d2cc9cfee6d481c20,SystemUUID:ec255807-a7de-c89d-2cc9-cfee6d481c20,BootID:4153aea0-5d07-4f37-a966-c56ef660acfc,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.1],SizeBytes:135178704,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.1],SizeBytes:124995897,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:63920476,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.1],SizeBytes:57661752,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:44339030,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42841305,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4965794,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:53:21.531: INFO:
Logging kubelet events for node i-0b0b3ac061164cea4
Jan 24 06:53:21.695: INFO:
Logging pods the kubelet thinks is on node i-0b0b3ac061164cea4
Jan 24 06:53:21.864: INFO: etcd-manager-main-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (10+1 container statuses recorded)
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Container etcd-manager ready: true, restart count 0
Jan 24 06:53:21.864: INFO: kube-controller-manager-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:21.864: INFO: Container kube-controller-manager ready: true, restart count 4
Jan 24 06:53:21.864: INFO: kube-scheduler-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:21.864: INFO: Container kube-scheduler ready: true, restart count 0
Jan 24 06:53:21.864: INFO: cilium-w225f started at 2023-01-24 06:26:07 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:21.864: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:53:21.864: INFO: cilium-operator-5dd44dc49f-rn9bl started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:21.864: INFO: Container cilium-operator ready: true, restart count 0
Jan 24 06:53:21.864: INFO: etcd-manager-events-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (10+1 container statuses recorded)
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Container etcd-manager ready: true, restart count 0
Jan 24 06:53:21.864: INFO: aws-cloud-controller-manager-4n9pv started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:21.864: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 24 06:53:21.864: INFO: ebs-csi-node-q4jsj started at 2023-01-24 06:26:07 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:21.864: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:21.864: INFO: kops-controller-vfj77 started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:21.864: INFO: Container kops-controller ready: true, restart count 0
Jan 24 06:53:21.864: INFO: dns-controller-69987775c6-vzqc8 started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:21.864: INFO: Container dns-controller ready: true, restart count 0
Jan 24 06:53:21.864: INFO: ebs-csi-controller-5bd98b456f-kswjq started at 2023-01-24 06:26:07 +0000 UTC (0+5 container statuses recorded)
Jan 24 06:53:21.864: INFO: Container csi-attacher ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Container csi-resizer ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:21.864: INFO: kube-apiserver-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+2 container statuses recorded)
Jan 24 06:53:21.864: INFO: Container healthcheck ready: true, restart count 0
Jan 24 06:53:21.864: INFO: Container kube-apiserver ready: true, restart count 3
Jan 24 06:53:22.422: INFO:
Latency metrics for node i-0b0b3ac061164cea4
Jan 24 06:53:22.422: INFO:
Logging node info for node i-0e0a289a9d5c333a0
Jan 24 06:53:22.584: INFO: Node Info: &Node{ObjectMeta:{i-0e0a289a9d5c333a0 9194794b-d78a-4a30-bbec-b76df63f68f1 33830 0 2023-01-24 06:27:40 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0e0a289a9d5c333a0 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0e0a289a9d5c333a0 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.62 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1968":"i-0e0a289a9d5c333a0","ebs.csi.aws.com":"i-0e0a289a9d5c333a0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:49:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0e0a289a9d5c333a0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.62,},NodeAddress{Type:ExternalIP,Address:54.180.158.26,},NodeAddress{Type:InternalDNS,Address:i-0e0a289a9d5c333a0.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0e0a289a9d5c333a0.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-180-158-26.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2281a7b00c77d27e8055a22bf9facf,SystemUUID:ec2281a7-b00c-77d2-7e80-55a22bf9facf,BootID:be2e68ba-6839-4b53-847f-22f29f0a7360,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1968^3d6e646f-9bb3-11ed-bc6b-56b051ebca30],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:53:22.585: INFO:
Logging kubelet events for node i-0e0a289a9d5c333a0
Jan 24 06:53:22.749: INFO:
Logging pods the kubelet thinks is on node i-0e0a289a9d5c333a0
Jan 24 06:53:22.920: INFO: test-ss-0 started at 2023-01-24 06:49:10 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container webserver ready: false, restart count 0
Jan 24 06:53:22.920: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-9hr5l started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:22.920: INFO: e2e-test-httpd-pod started at 2023-01-24 06:49:16 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container e2e-test-httpd-pod ready: false, restart count 0
Jan 24 06:53:22.920: INFO: pod1 started at 2023-01-24 06:50:50 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container agnhost-container ready: false, restart count 0
Jan 24 06:53:22.920: INFO: service-headless-h4z88 started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container service-headless ready: true, restart count 0
Jan 24 06:53:22.920: INFO: dns-test-b3d6c428-d0cf-4d5a-9700-fdfb874b5d91 started at 2023-01-24 06:48:41 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container jessie-querier ready: false, restart count 0
Jan 24 06:53:22.920: INFO: Container querier ready: false, restart count 0
Jan 24 06:53:22.920: INFO: Container webserver ready: false, restart count 0
Jan 24 06:53:22.920: INFO: cilium-cvt86 started at 2023-01-24 06:27:41 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:53:22.920: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:53:22.920: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-224qm started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:22.920: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-z9lc4 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:22.920: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-7nhf8 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:22.920: INFO: rs-l5fgd started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:22.920: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-867tg started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:22.920: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-zv4q6 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:22.920: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-tbbms started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:22.920: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-w7m2k started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:53:22.920: INFO: ebs-csi-node-v6prc started at 2023-01-24 06:27:41 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:22.920: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:22.920: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:22.920: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-ts8xr started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:22.920: INFO: rs-rsqdt started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container donothing ready: true, restart count 0
Jan 24 06:53:22.920: INFO: rc-test-qrc82 started at 2023-01-24 06:48:11 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container rc-test ready: false, restart count 0
Jan 24 06:53:22.920: INFO: pvc-volume-tester-56wjf started at 2023-01-24 06:49:25 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container volume-tester ready: false, restart count 0
Jan 24 06:53:22.920: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-bkjzk started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:22.920: INFO: csi-mockplugin-0 started at 2023-01-24 06:48:18 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:53:22.920: INFO: Container driver-registrar ready: true, restart count 0
Jan 24 06:53:22.920: INFO: Container mock ready: true, restart count 0
Jan 24 06:53:22.920: INFO: hostexec-i-0e0a289a9d5c333a0-7tx4h started at 2023-01-24 06:53:15 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container agnhost-container ready: true, restart count 0
Jan 24 06:53:22.920: INFO: rs-znpr8 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:22.920: INFO: tester started at 2023-01-24 06:53:16 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container tester ready: false, restart count 0
Jan 24 06:53:22.920: INFO: startup-e83c2b2e-e8d4-4137-b286-5a3397f52961 started at 2023-01-24 06:49:02 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container busybox ready: false, restart count 0
Jan 24 06:53:22.920: INFO: pod-1 started at 2023-01-24 06:53:19 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:22.920: INFO: csi-mockplugin-attacher-0 started at 2023-01-24 06:48:19 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:22.920: INFO: Container csi-attacher ready: false, restart count 0
Jan 24 06:53:23.516: INFO:
Latency metrics for node i-0e0a289a9d5c333a0
Jan 24 06:53:23.516: INFO:
Logging node info for node i-0f5ee4c33b6799e76
Jan 24 06:53:23.678: INFO: Node Info: &Node{ObjectMeta:{i-0f5ee4c33b6799e76 34c5957d-82ce-41f2-824d-90a78fa8bbba 33946 0 2023-01-24 06:27:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0f5ee4c33b6799e76 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0f5ee4c33b6799e76 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.57.132 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0f5ee4c33b6799e76"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:49:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0f5ee4c33b6799e76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.57.132,},NodeAddress{Type:ExternalIP,Address:3.36.124.131,},NodeAddress{Type:InternalDNS,Address:i-0f5ee4c33b6799e76.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f5ee4c33b6799e76.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-36-124-131.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bcf8e071d2b94271b14b41c978c,SystemUUID:ec2f5bcf-8e07-1d2b-9427-1b14b41c978c,BootID:400ceb00-7274-4349-89f8-a3cb13aed9c2,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:53:23.679: INFO:
Logging kubelet events for node i-0f5ee4c33b6799e76
Jan 24 06:53:23.843: INFO:
Logging pods the kubelet thinks is on node i-0f5ee4c33b6799e76
Jan 24 06:53:24.012: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-w9729 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:24.012: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-lvvw2 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:24.012: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-wjt45 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:24.012: INFO: coredns-559769c974-5gltc started at 2023-01-24 06:28:16 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container coredns ready: true, restart count 0
Jan 24 06:53:24.012: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-7lw99 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:24.012: INFO: ebs-csi-node-vjhdb started at 2023-01-24 06:27:40 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:24.012: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:24.012: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:24.012: INFO: pod-failure-failjob-s22zv started at 2023-01-24 06:49:00 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container c ready: false, restart count 0
Jan 24 06:53:24.012: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-r5vgx started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:24.012: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6lkqq started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:24.012: INFO: pod-failure-failjob-s2mc2 started at 2023-01-24 06:49:00 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container c ready: false, restart count 0
Jan 24 06:53:24.012: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qmz8q started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:24.012: INFO: rs-g6rhr started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container donothing ready: true, restart count 0
Jan 24 06:53:24.012: INFO: rs-8b8n9 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:24.012: INFO: cilium-ql489 started at 2023-01-24 06:27:40 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:53:24.012: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:53:24.012: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-hwktw started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:24.012: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cfnfh started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:24.012: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-xl6bg started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:24.012: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:24.597: INFO:
Latency metrics for node i-0f5ee4c33b6799e76
[DeferCleanup (Each)] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "fsgroupchangepolicy-4417" for this suite. �[38;5;243m01/24/23 06:53:24.597�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-apps\]\sReplicationController\sshould\stest\sthe\slifecycle\sof\sa\sReplicationController\s\[Conformance\]$'
test/e2e/apps/rc.go:189
k8s.io/kubernetes/test/e2e/apps.glob..func8.7.1(0xc0032c4160?)
test/e2e/apps/rc.go:189 +0x49d
k8s.io/kubernetes/test/e2e/framework.WatchEventSequenceVerifier({0x7fe5ba8?, 0xc000132008}, {0x7fb1880?, _}, {{0x0, 0x0}, {0x75b9f5c, 0x2}, {0x7614288, 0x16}}, ...)
test/e2e/framework/util.go:721 +0x575
k8s.io/kubernetes/test/e2e/apps.glob..func8.7()
test/e2e/apps/rc.go:148 +0x753
[BeforeEach] [sig-apps] ReplicationController
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/24/23 06:48:10.624�[0m
Jan 24 06:48:10.624: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename replication-controller �[38;5;243m01/24/23 06:48:10.626�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/24/23 06:48:11.093�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/24/23 06:48:11.404�[0m
[BeforeEach] [sig-apps] ReplicationController
test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-apps] ReplicationController
test/e2e/apps/rc.go:57
[It] should test the lifecycle of a ReplicationController [Conformance]
test/e2e/apps/rc.go:110
�[1mSTEP:�[0m creating a ReplicationController �[38;5;243m01/24/23 06:48:11.869�[0m
�[1mSTEP:�[0m waiting for RC to be added �[38;5;243m01/24/23 06:48:12.03�[0m
�[1mSTEP:�[0m waiting for available Replicas �[38;5;243m01/24/23 06:48:12.03�[0m
Jan 24 06:53:12.031: INFO: Unexpected error: Wait for condition with watch events should not return an error:
<*errors.errorString | 0xc0001c39f0>: {
s: "timed out waiting for the condition",
}
Jan 24 06:53:12.031: FAIL: Wait for condition with watch events should not return an error: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/apps.glob..func8.7.1(0xc0032c4160?)
test/e2e/apps/rc.go:189 +0x49d
k8s.io/kubernetes/test/e2e/framework.WatchEventSequenceVerifier({0x7fe5ba8?, 0xc000132008}, {0x7fb1880?, _}, {{0x0, 0x0}, {0x75b9f5c, 0x2}, {0x7614288, 0x16}}, ...)
test/e2e/framework/util.go:721 +0x575
k8s.io/kubernetes/test/e2e/apps.glob..func8.7()
test/e2e/apps/rc.go:148 +0x753
[AfterEach] [sig-apps] ReplicationController
test/e2e/framework/node/init/init.go:32
Jan 24 06:53:12.032: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-apps] ReplicationController
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-apps] ReplicationController
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/24/23 06:53:12.188�[0m
�[1mSTEP:�[0m Collecting events from namespace "replication-controller-209". �[38;5;243m01/24/23 06:53:12.189�[0m
�[1mSTEP:�[0m Found 12 events. �[38;5;243m01/24/23 06:53:12.344�[0m
Jan 24 06:53:12.344: INFO: At 2023-01-24 06:48:11 +0000 UTC - event for rc-test: {replication-controller } SuccessfulCreate: Created pod: rc-test-qrc82
Jan 24 06:53:12.344: INFO: At 2023-01-24 06:48:11 +0000 UTC - event for rc-test-qrc82: {default-scheduler } Scheduled: Successfully assigned replication-controller-209/rc-test-qrc82 to i-0e0a289a9d5c333a0
Jan 24 06:53:12.344: INFO: At 2023-01-24 06:48:12 +0000 UTC - event for rc-test-qrc82: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7e7067567c543c251255c6124d4fb0070bf40d50fea7494fdb720da53bf5ee0b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:12.344: INFO: At 2023-01-24 06:48:26 +0000 UTC - event for rc-test-qrc82: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4f808acb76cc2730686d89e4edce3c73285a60fde22275162daa26d98c05c1c8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:12.344: INFO: At 2023-01-24 06:48:37 +0000 UTC - event for rc-test-qrc82: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c8289108b085f6df17e7b5ca3a465ececc25eb1480fce8c544ce08a62d9705ff": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:12.344: INFO: At 2023-01-24 06:48:50 +0000 UTC - event for rc-test-qrc82: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "48b3cf00f3143ce726c8525bff57aad0d86d302ecfa7053c7d6e6fb3c131e647": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:12.344: INFO: At 2023-01-24 06:49:03 +0000 UTC - event for rc-test-qrc82: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "371e56c90abb97ec1d2a208ed772732ee7bc95a462d5f55dc076f2494a9e8777": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:12.344: INFO: At 2023-01-24 06:49:18 +0000 UTC - event for rc-test-qrc82: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f6e94feebdc1c469d610ede4c407edb6ea3f24c0ee85f3322d170ed938f4c5ce": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:12.344: INFO: At 2023-01-24 06:49:32 +0000 UTC - event for rc-test-qrc82: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a6a1c1bb3c46c7b4a0cf6096d276d712f37f1891403f3fc6ebbf9267fdb85ce6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:12.344: INFO: At 2023-01-24 06:49:44 +0000 UTC - event for rc-test-qrc82: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6dd930ca19309adfd8b1365b1def57eb5771ea29c03e08383227c4b9bab4713f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:12.344: INFO: At 2023-01-24 06:49:56 +0000 UTC - event for rc-test-qrc82: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "df931cc98a1dccf81c476aa0f8e4083fbc9b3fe3032d737c04073a16ff32032c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:12.344: INFO: At 2023-01-24 06:50:08 +0000 UTC - event for rc-test-qrc82: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a6bd96661c39f898b23c5730ee246edf4b7a3c34fb3b9a79b30f6d81f4eb4205": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:12.500: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 24 06:53:12.500: INFO: rc-test-qrc82 i-0e0a289a9d5c333a0 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:48:11 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:48:11 +0000 UTC ContainersNotReady containers with unready status: [rc-test]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:48:11 +0000 UTC ContainersNotReady containers with unready status: [rc-test]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:48:11 +0000 UTC }]
Jan 24 06:53:12.500: INFO:
Jan 24 06:53:12.662: INFO: Unable to fetch replication-controller-209/rc-test-qrc82/rc-test logs: the server rejected our request for an unknown reason (get pods rc-test-qrc82)
Jan 24 06:53:12.819: INFO:
Logging node info for node i-036f641c68d5ad392
Jan 24 06:53:12.975: INFO: Node Info: &Node{ObjectMeta:{i-036f641c68d5ad392 50721840-3696-4207-81a9-8cd089fbb539 33635 0 2023-01-24 06:27:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-036f641c68d5ad392 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-036f641c68d5ad392 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.254 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-5409":"i-036f641c68d5ad392","ebs.csi.aws.com":"i-036f641c68d5ad392"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:48:57 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-24 06:49:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-036f641c68d5ad392,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:04 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:04 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:04 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:49:04 +0000 UTC,LastTransitionTime:2023-01-24 06:27:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.254,},NodeAddress{Type:ExternalIP,Address:15.165.237.223,},NodeAddress{Type:InternalDNS,Address:i-036f641c68d5ad392.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-036f641c68d5ad392.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-165-237-223.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2732c784a9bb7e64a787956c82cf85,SystemUUID:ec2732c7-84a9-bb7e-64a7-87956c82cf85,BootID:bee5ed40-42b1-475e-b72b-16d715de535a,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-5409^2c26217c-9bb3-11ed-bce5-86d6f2bc551f kubernetes.io/csi/ebs.csi.aws.com^vol-0269089ae9a4173b7],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0269089ae9a4173b7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-5409^2c26217c-9bb3-11ed-bce5-86d6f2bc551f,DevicePath:,},},Config:nil,},}
Jan 24 06:53:12.975: INFO:
Logging kubelet events for node i-036f641c68d5ad392
Jan 24 06:53:13.133: INFO:
Logging pods the kubelet thinks is on node i-036f641c68d5ad392
Jan 24 06:53:13.298: INFO: pfpod started at 2023-01-24 06:48:43 +0000 UTC (0+2 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container portforwardtester ready: false, restart count 0
Jan 24 06:53:13.298: INFO: Container readiness ready: false, restart count 0
Jan 24 06:53:13.298: INFO: dns-test-93a0bf56-7463-4d4f-a334-bb3e44707aaa started at 2023-01-24 06:51:48 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container jessie-querier ready: false, restart count 0
Jan 24 06:53:13.298: INFO: Container querier ready: false, restart count 0
Jan 24 06:53:13.298: INFO: Container webserver ready: false, restart count 0
Jan 24 06:53:13.298: INFO: rs-76gck started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:13.298: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-m4sr7 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:13.298: INFO: csi-mockplugin-0 started at 2023-01-24 06:47:12 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container csi-provisioner ready: false, restart count 0
Jan 24 06:53:13.298: INFO: Container driver-registrar ready: false, restart count 0
Jan 24 06:53:13.298: INFO: Container mock ready: false, restart count 0
Jan 24 06:53:13.298: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6twph started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:13.298: INFO: rs-6s4df started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:13.298: INFO: csi-hostpathplugin-0 started at 2023-01-24 06:44:58 +0000 UTC (0+7 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container csi-attacher ready: true, restart count 0
Jan 24 06:53:13.298: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:53:13.298: INFO: Container csi-resizer ready: true, restart count 0
Jan 24 06:53:13.298: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 24 06:53:13.298: INFO: Container hostpath ready: true, restart count 0
Jan 24 06:53:13.298: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:13.298: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:13.298: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2mkph started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:13.298: INFO: rs-zmrj2 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:13.298: INFO: pod-projected-configmaps-495807fd-45a6-4d3f-a257-d364553fb370 started at 2023-01-24 06:48:40 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container agnhost-container ready: false, restart count 0
Jan 24 06:53:13.298: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-8sfzb started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:13.298: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cpp7h started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:13.298: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6hnbl started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:13.298: INFO: service-headless-btk74 started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container service-headless ready: false, restart count 0
Jan 24 06:53:13.298: INFO: pod-da9584f9-0d79-4167-ad37-ca206c3ac911 started at 2023-01-24 06:43:12 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container write-pod ready: false, restart count 0
Jan 24 06:53:13.298: INFO: ebs-csi-node-6z65r started at 2023-01-24 06:27:40 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:13.298: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:13.298: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:13.298: INFO: pod-handle-http-request started at 2023-01-24 06:48:15 +0000 UTC (0+2 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container container-handle-http-request ready: false, restart count 0
Jan 24 06:53:13.298: INFO: Container container-handle-https-request ready: false, restart count 0
Jan 24 06:53:13.298: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-h5mpm started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:13.298: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-tcp26 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:13.298: INFO: csi-mockplugin-resizer-0 started at 2023-01-24 06:47:12 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container csi-resizer ready: true, restart count 0
Jan 24 06:53:13.298: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-t9b6f started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:13.298: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-lc7s4 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:13.298: INFO: rand-non-local-gxd9p started at 2023-01-24 06:42:31 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container c ready: false, restart count 0
Jan 24 06:53:13.298: INFO: cilium-9ht2g started at 2023-01-24 06:27:40 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:53:13.298: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:53:13.298: INFO: server started at 2023-01-24 06:49:10 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container agnhost-container ready: false, restart count 0
Jan 24 06:53:13.298: INFO: test-ss-0 started at 2023-01-24 06:49:23 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container webserver ready: false, restart count 0
Jan 24 06:53:13.298: INFO: pod-configmaps-5f55ab46-5e11-4c96-87a9-f170ab7204ce started at 2023-01-24 06:49:28 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container agnhost-container ready: false, restart count 0
Jan 24 06:53:13.298: INFO: pod-56c7bad9-2975-4685-aab0-0cfb8b42331c started at 2023-01-24 06:48:56 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:13.298: INFO: Container write-pod ready: false, restart count 0
Jan 24 06:53:14.035: INFO:
Latency metrics for node i-036f641c68d5ad392
Jan 24 06:53:14.035: INFO:
Logging node info for node i-0697a70b9aa95b7c0
Jan 24 06:53:14.191: INFO: Node Info: &Node{ObjectMeta:{i-0697a70b9aa95b7c0 d0a3c976-eb07-4323-b48b-ec4945de7410 33229 0 2023-01-24 06:27:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0697a70b9aa95b7c0 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0697a70b9aa95b7c0 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.153 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0697a70b9aa95b7c0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:48:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0697a70b9aa95b7c0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:49 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.153,},NodeAddress{Type:ExternalIP,Address:3.38.96.49,},NodeAddress{Type:InternalDNS,Address:i-0697a70b9aa95b7c0.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0697a70b9aa95b7c0.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-38-96-49.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f1e14d470f251c9d042c57478cb8b,SystemUUID:ec2f1e14-d470-f251-c9d0-42c57478cb8b,BootID:578a4c1d-084e-4aeb-92fe-f64671842d82,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:53:14.191: INFO:
Logging kubelet events for node i-0697a70b9aa95b7c0
Jan 24 06:53:14.350: INFO:
Logging pods the kubelet thinks is on node i-0697a70b9aa95b7c0
Jan 24 06:53:14.513: INFO: cilium-dh7hm started at 2023-01-24 06:27:36 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:53:14.513: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:53:14.513: INFO: coredns-559769c974-6wblg started at 2023-01-24 06:27:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container coredns ready: true, restart count 0
Jan 24 06:53:14.513: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-5pcw5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:14.513: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cdkdr started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:14.513: INFO: rs-w9wjl started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container donothing ready: true, restart count 0
Jan 24 06:53:14.513: INFO: pod-subpath-test-preprovisionedpv-94gx started at 2023-01-24 06:50:18 +0000 UTC (2+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Init container init-volume-preprovisionedpv-94gx ready: false, restart count 0
Jan 24 06:53:14.513: INFO: Init container test-init-volume-preprovisionedpv-94gx ready: false, restart count 0
Jan 24 06:53:14.513: INFO: Container test-container-subpath-preprovisionedpv-94gx ready: false, restart count 0
Jan 24 06:53:14.513: INFO: coredns-autoscaler-7cb5c5b969-vktlf started at 2023-01-24 06:27:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container autoscaler ready: true, restart count 0
Jan 24 06:53:14.513: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-xbpd8 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:53:14.513: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-4sk4h started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:14.513: INFO: hostexec-i-0697a70b9aa95b7c0-n9ccb started at 2023-01-24 06:49:56 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container agnhost-container ready: true, restart count 0
Jan 24 06:53:14.513: INFO: ebs-csi-node-pkrr4 started at 2023-01-24 06:27:36 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:14.513: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:14.513: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:14.513: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-fgc56 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:14.513: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2v7c5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:14.513: INFO: service-headless-xh4pl started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container service-headless ready: false, restart count 0
Jan 24 06:53:14.513: INFO: csi-mockplugin-0 started at 2023-01-24 06:48:54 +0000 UTC (0+4 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container busybox ready: false, restart count 0
Jan 24 06:53:14.513: INFO: Container csi-provisioner ready: false, restart count 0
Jan 24 06:53:14.513: INFO: Container driver-registrar ready: false, restart count 0
Jan 24 06:53:14.513: INFO: Container mock ready: false, restart count 0
Jan 24 06:53:14.513: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-68w92 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:14.513: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qphds started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:14.513: INFO: rs-b9vhw started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container donothing ready: true, restart count 0
Jan 24 06:53:14.513: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2tkvs started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:53:14.513: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qtzq5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:14.513: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:15.081: INFO:
Latency metrics for node i-0697a70b9aa95b7c0
Jan 24 06:53:15.081: INFO:
Logging node info for node i-0b0b3ac061164cea4
Jan 24 06:53:15.237: INFO: Node Info: &Node{ObjectMeta:{i-0b0b3ac061164cea4 ac00aaf7-4427-49f6-b849-9c812ae80069 33008 0 2023-01-24 06:25:23 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b0b3ac061164cea4 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.51.48 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b0b3ac061164cea4"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-24 06:25:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-24 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:26:22 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:26:22 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-24 06:26:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-24 06:48:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0b0b3ac061164cea4,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892297728 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787440128 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:26:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.48,},NodeAddress{Type:ExternalIP,Address:13.125.222.159,},NodeAddress{Type:InternalDNS,Address:i-0b0b3ac061164cea4.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b0b3ac061164cea4.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-125-222-159.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255807a7dec89d2cc9cfee6d481c20,SystemUUID:ec255807-a7de-c89d-2cc9-cfee6d481c20,BootID:4153aea0-5d07-4f37-a966-c56ef660acfc,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.1],SizeBytes:135178704,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.1],SizeBytes:124995897,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:63920476,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.1],SizeBytes:57661752,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:44339030,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42841305,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4965794,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:53:15.237: INFO:
Logging kubelet events for node i-0b0b3ac061164cea4
Jan 24 06:53:15.395: INFO:
Logging pods the kubelet thinks is on node i-0b0b3ac061164cea4
Jan 24 06:53:15.557: INFO: kops-controller-vfj77 started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:15.557: INFO: Container kops-controller ready: true, restart count 0
Jan 24 06:53:15.557: INFO: dns-controller-69987775c6-vzqc8 started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:15.557: INFO: Container dns-controller ready: true, restart count 0
Jan 24 06:53:15.557: INFO: ebs-csi-controller-5bd98b456f-kswjq started at 2023-01-24 06:26:07 +0000 UTC (0+5 container statuses recorded)
Jan 24 06:53:15.557: INFO: Container csi-attacher ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Container csi-resizer ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:15.557: INFO: kube-apiserver-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+2 container statuses recorded)
Jan 24 06:53:15.557: INFO: Container healthcheck ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Container kube-apiserver ready: true, restart count 3
Jan 24 06:53:15.557: INFO: aws-cloud-controller-manager-4n9pv started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:15.557: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 24 06:53:15.557: INFO: ebs-csi-node-q4jsj started at 2023-01-24 06:26:07 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:15.557: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:15.557: INFO: kube-scheduler-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:15.557: INFO: Container kube-scheduler ready: true, restart count 0
Jan 24 06:53:15.557: INFO: cilium-w225f started at 2023-01-24 06:26:07 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:15.557: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:53:15.557: INFO: cilium-operator-5dd44dc49f-rn9bl started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:15.557: INFO: Container cilium-operator ready: true, restart count 0
Jan 24 06:53:15.557: INFO: etcd-manager-events-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (10+1 container statuses recorded)
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Container etcd-manager ready: true, restart count 0
Jan 24 06:53:15.557: INFO: etcd-manager-main-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (10+1 container statuses recorded)
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 24 06:53:15.557: INFO: Container etcd-manager ready: true, restart count 0
Jan 24 06:53:15.557: INFO: kube-controller-manager-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:15.557: INFO: Container kube-controller-manager ready: true, restart count 4
Jan 24 06:53:16.081: INFO:
Latency metrics for node i-0b0b3ac061164cea4
Jan 24 06:53:16.081: INFO:
Logging node info for node i-0e0a289a9d5c333a0
Jan 24 06:53:16.237: INFO: Node Info: &Node{ObjectMeta:{i-0e0a289a9d5c333a0 9194794b-d78a-4a30-bbec-b76df63f68f1 33830 0 2023-01-24 06:27:40 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0e0a289a9d5c333a0 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0e0a289a9d5c333a0 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.62 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1968":"i-0e0a289a9d5c333a0","ebs.csi.aws.com":"i-0e0a289a9d5c333a0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:49:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0e0a289a9d5c333a0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.62,},NodeAddress{Type:ExternalIP,Address:54.180.158.26,},NodeAddress{Type:InternalDNS,Address:i-0e0a289a9d5c333a0.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0e0a289a9d5c333a0.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-180-158-26.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2281a7b00c77d27e8055a22bf9facf,SystemUUID:ec2281a7-b00c-77d2-7e80-55a22bf9facf,BootID:be2e68ba-6839-4b53-847f-22f29f0a7360,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1968^3d6e646f-9bb3-11ed-bc6b-56b051ebca30],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:53:16.238: INFO:
Logging kubelet events for node i-0e0a289a9d5c333a0
Jan 24 06:53:16.396: INFO:
Logging pods the kubelet thinks is on node i-0e0a289a9d5c333a0
Jan 24 06:53:16.561: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-867tg started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:16.561: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-zv4q6 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:16.561: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-tbbms started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:16.561: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-w7m2k started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:53:16.561: INFO: ebs-csi-node-v6prc started at 2023-01-24 06:27:41 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:16.561: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:16.561: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:16.561: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-ts8xr started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:16.561: INFO: rs-rsqdt started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container donothing ready: true, restart count 0
Jan 24 06:53:16.561: INFO: rc-test-qrc82 started at 2023-01-24 06:48:11 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container rc-test ready: false, restart count 0
Jan 24 06:53:16.561: INFO: pvc-volume-tester-56wjf started at 2023-01-24 06:49:25 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container volume-tester ready: false, restart count 0
Jan 24 06:53:16.561: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-bkjzk started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:16.561: INFO: csi-mockplugin-0 started at 2023-01-24 06:48:18 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:53:16.561: INFO: Container driver-registrar ready: true, restart count 0
Jan 24 06:53:16.561: INFO: Container mock ready: true, restart count 0
Jan 24 06:53:16.561: INFO: hostexec-i-0e0a289a9d5c333a0-7tx4h started at 2023-01-24 06:53:15 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container agnhost-container ready: false, restart count 0
Jan 24 06:53:16.561: INFO: rs-znpr8 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:16.561: INFO: startup-e83c2b2e-e8d4-4137-b286-5a3397f52961 started at 2023-01-24 06:49:02 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container busybox ready: false, restart count 0
Jan 24 06:53:16.561: INFO: csi-mockplugin-attacher-0 started at 2023-01-24 06:48:19 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container csi-attacher ready: false, restart count 0
Jan 24 06:53:16.561: INFO: test-ss-0 started at 2023-01-24 06:49:10 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container webserver ready: false, restart count 0
Jan 24 06:53:16.561: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-9hr5l started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:16.561: INFO: e2e-test-httpd-pod started at 2023-01-24 06:49:16 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container e2e-test-httpd-pod ready: false, restart count 0
Jan 24 06:53:16.561: INFO: pod1 started at 2023-01-24 06:50:50 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container agnhost-container ready: false, restart count 0
Jan 24 06:53:16.561: INFO: service-headless-h4z88 started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container service-headless ready: true, restart count 0
Jan 24 06:53:16.561: INFO: dns-test-b3d6c428-d0cf-4d5a-9700-fdfb874b5d91 started at 2023-01-24 06:48:41 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container jessie-querier ready: false, restart count 0
Jan 24 06:53:16.561: INFO: Container querier ready: false, restart count 0
Jan 24 06:53:16.561: INFO: Container webserver ready: false, restart count 0
Jan 24 06:53:16.561: INFO: cilium-cvt86 started at 2023-01-24 06:27:41 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:53:16.561: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:53:16.561: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-224qm started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:16.561: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-z9lc4 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:16.561: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-7nhf8 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:16.561: INFO: rs-l5fgd started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:16.561: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:17.244: INFO:
Latency metrics for node i-0e0a289a9d5c333a0
Jan 24 06:53:17.244: INFO:
Logging node info for node i-0f5ee4c33b6799e76
Jan 24 06:53:17.400: INFO: Node Info: &Node{ObjectMeta:{i-0f5ee4c33b6799e76 34c5957d-82ce-41f2-824d-90a78fa8bbba 33946 0 2023-01-24 06:27:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0f5ee4c33b6799e76 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0f5ee4c33b6799e76 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.57.132 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0f5ee4c33b6799e76"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:49:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0f5ee4c33b6799e76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.57.132,},NodeAddress{Type:ExternalIP,Address:3.36.124.131,},NodeAddress{Type:InternalDNS,Address:i-0f5ee4c33b6799e76.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f5ee4c33b6799e76.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-36-124-131.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bcf8e071d2b94271b14b41c978c,SystemUUID:ec2f5bcf-8e07-1d2b-9427-1b14b41c978c,BootID:400ceb00-7274-4349-89f8-a3cb13aed9c2,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:53:17.400: INFO:
Logging kubelet events for node i-0f5ee4c33b6799e76
Jan 24 06:53:17.558: INFO:
Logging pods the kubelet thinks is on node i-0f5ee4c33b6799e76
Jan 24 06:53:17.721: INFO: cilium-ql489 started at 2023-01-24 06:27:40 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:53:17.721: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:53:17.721: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-hwktw started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:17.721: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cfnfh started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:17.721: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-xl6bg started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:17.721: INFO: rs-g6rhr started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container donothing ready: true, restart count 0
Jan 24 06:53:17.721: INFO: rs-8b8n9 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:17.721: INFO: coredns-559769c974-5gltc started at 2023-01-24 06:28:16 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container coredns ready: true, restart count 0
Jan 24 06:53:17.721: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-7lw99 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:17.721: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-w9729 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:17.721: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-lvvw2 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:17.721: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-wjt45 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:17.721: INFO: ebs-csi-node-vjhdb started at 2023-01-24 06:27:40 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:17.721: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:17.721: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:17.721: INFO: pod-failure-failjob-s22zv started at 2023-01-24 06:49:00 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container c ready: false, restart count 0
Jan 24 06:53:17.721: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-r5vgx started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:17.721: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6lkqq started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:17.721: INFO: pod-failure-failjob-s2mc2 started at 2023-01-24 06:49:00 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container c ready: false, restart count 0
Jan 24 06:53:17.721: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qmz8q started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:17.721: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:18.290: INFO:
Latency metrics for node i-0f5ee4c33b6799e76
[DeferCleanup (Each)] [sig-apps] ReplicationController
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "replication-controller-209" for this suite. �[38;5;243m01/24/23 06:53:18.29�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sPort\sforwarding\sWith\sa\sserver\slistening\son\slocalhost\sthat\sexpects\sNO\sclient\srequest\sshould\ssupport\sa\sclient\sthat\sconnects\,\ssends\sDATA\,\sand\sdisconnects$'
test/e2e/kubectl/portforward.go:217
k8s.io/kubernetes/test/e2e/kubectl.doTestConnectSendDisconnect({0x75cb648, 0x9}, 0xc000f543c0)
test/e2e/kubectl/portforward.go:217 +0x24c
k8s.io/kubernetes/test/e2e/kubectl.glob..func2.2.2.1()
test/e2e/kubectl/portforward.go:488 +0x29
[BeforeEach] [sig-cli] Kubectl Port forwarding
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/24/23 06:48:42.536�[0m
Jan 24 06:48:42.536: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename port-forwarding �[38;5;243m01/24/23 06:48:42.537�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/24/23 06:48:43.016�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/24/23 06:48:43.333�[0m
[BeforeEach] [sig-cli] Kubectl Port forwarding
test/e2e/framework/metrics/init/init.go:31
[It] should support a client that connects, sends DATA, and disconnects
test/e2e/kubectl/portforward.go:487
�[1mSTEP:�[0m Creating the target pod �[38;5;243m01/24/23 06:48:43.651�[0m
Jan 24 06:48:43.814: INFO: Waiting up to 5m0s for pod "pfpod" in namespace "port-forwarding-5834" to be "running and ready"
Jan 24 06:48:43.974: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 159.497625ms
Jan 24 06:48:43.974: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:46.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2.319983386s
Jan 24 06:48:46.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:48.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4.319927267s
Jan 24 06:48:48.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:50.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 6.320194788s
Jan 24 06:48:50.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:52.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 8.319888008s
Jan 24 06:48:52.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:54.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 10.319761343s
Jan 24 06:48:54.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:56.136: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 12.32124461s
Jan 24 06:48:56.136: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:58.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 14.320147939s
Jan 24 06:48:58.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:00.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 16.319734509s
Jan 24 06:49:00.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:02.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 18.319783328s
Jan 24 06:49:02.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:04.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 20.31960441s
Jan 24 06:49:04.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:06.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 22.319874426s
Jan 24 06:49:06.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:08.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 24.31990905s
Jan 24 06:49:08.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:10.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 26.319852106s
Jan 24 06:49:10.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:12.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 28.320014349s
Jan 24 06:49:12.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:14.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 30.31976893s
Jan 24 06:49:14.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:16.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 32.319402127s
Jan 24 06:49:16.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:18.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 34.320122764s
Jan 24 06:49:18.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:20.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 36.319725971s
Jan 24 06:49:20.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:22.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 38.319782896s
Jan 24 06:49:22.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:24.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 40.320491471s
Jan 24 06:49:24.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:26.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 42.319867488s
Jan 24 06:49:26.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:28.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 44.319894889s
Jan 24 06:49:28.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:30.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 46.320076927s
Jan 24 06:49:30.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:32.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 48.319788941s
Jan 24 06:49:32.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:34.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 50.319886055s
Jan 24 06:49:34.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:36.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 52.319852206s
Jan 24 06:49:36.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:38.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 54.320099005s
Jan 24 06:49:38.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:40.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 56.319650301s
Jan 24 06:49:40.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:42.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 58.319843806s
Jan 24 06:49:42.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:44.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.319696182s
Jan 24 06:49:44.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:46.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.319800667s
Jan 24 06:49:46.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:48.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.320102821s
Jan 24 06:49:48.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:50.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.319323575s
Jan 24 06:49:50.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:52.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.319759586s
Jan 24 06:49:52.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:54.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.320122362s
Jan 24 06:49:54.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:56.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.31960725s
Jan 24 06:49:56.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:58.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.31941271s
Jan 24 06:49:58.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:00.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.320042317s
Jan 24 06:50:00.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:02.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.319916334s
Jan 24 06:50:02.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:04.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.320025191s
Jan 24 06:50:04.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:06.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.319989281s
Jan 24 06:50:06.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:08.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.319846305s
Jan 24 06:50:08.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:10.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.320036115s
Jan 24 06:50:10.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:12.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.320355525s
Jan 24 06:50:12.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:14.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.319935339s
Jan 24 06:50:14.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:16.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.319787584s
Jan 24 06:50:16.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:18.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.319811388s
Jan 24 06:50:18.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:20.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.319859063s
Jan 24 06:50:20.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:22.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.319685613s
Jan 24 06:50:22.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:24.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.319951774s
Jan 24 06:50:24.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:26.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.32007436s
Jan 24 06:50:26.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:28.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.31972361s
Jan 24 06:50:28.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:30.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.319757991s
Jan 24 06:50:30.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:32.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.319826043s
Jan 24 06:50:32.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:34.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.319647163s
Jan 24 06:50:34.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:36.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.319996616s
Jan 24 06:50:36.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:38.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.3203445s
Jan 24 06:50:38.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:40.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.319962464s
Jan 24 06:50:40.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:42.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.319240296s
Jan 24 06:50:42.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:44.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.319896425s
Jan 24 06:50:44.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:46.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.320199372s
Jan 24 06:50:46.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:48.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.320132031s
Jan 24 06:50:48.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:50.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.320172297s
Jan 24 06:50:50.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:52.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.319952075s
Jan 24 06:50:52.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:54.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.320062923s
Jan 24 06:50:54.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:56.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.319300234s
Jan 24 06:50:56.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:50:58.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.319953986s
Jan 24 06:50:58.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:00.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.32028437s
Jan 24 06:51:00.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:02.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.319910535s
Jan 24 06:51:02.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:04.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.319948998s
Jan 24 06:51:04.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:06.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.319904627s
Jan 24 06:51:06.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:08.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.320028268s
Jan 24 06:51:08.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:10.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.3202133s
Jan 24 06:51:10.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:12.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.319901261s
Jan 24 06:51:12.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:14.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.319747407s
Jan 24 06:51:14.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:16.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.319946766s
Jan 24 06:51:16.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:18.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.31991346s
Jan 24 06:51:18.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:20.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.319880387s
Jan 24 06:51:20.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:22.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.319753602s
Jan 24 06:51:22.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:24.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.319520521s
Jan 24 06:51:24.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:26.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.319997673s
Jan 24 06:51:26.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:28.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.319788292s
Jan 24 06:51:28.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:30.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.320063775s
Jan 24 06:51:30.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:32.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.31957875s
Jan 24 06:51:32.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:34.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.320561896s
Jan 24 06:51:34.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:36.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.320110386s
Jan 24 06:51:36.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:38.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.319962459s
Jan 24 06:51:38.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:40.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.319919348s
Jan 24 06:51:40.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:42.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.320008614s
Jan 24 06:51:42.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:44.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.320011026s
Jan 24 06:51:44.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:46.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.319960684s
Jan 24 06:51:46.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:48.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.320177242s
Jan 24 06:51:48.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:50.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.320119418s
Jan 24 06:51:50.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:52.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.319301893s
Jan 24 06:51:52.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:54.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.320272399s
Jan 24 06:51:54.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:56.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.320101127s
Jan 24 06:51:56.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:51:58.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.319927679s
Jan 24 06:51:58.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:00.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.320280309s
Jan 24 06:52:00.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:02.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.319998547s
Jan 24 06:52:02.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:04.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.320142953s
Jan 24 06:52:04.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:06.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.320029646s
Jan 24 06:52:06.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:08.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.32009774s
Jan 24 06:52:08.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:10.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.319763102s
Jan 24 06:52:10.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:12.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.319302167s
Jan 24 06:52:12.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:14.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.319847676s
Jan 24 06:52:14.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:16.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.320386482s
Jan 24 06:52:16.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:18.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.319505637s
Jan 24 06:52:18.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:20.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.320104161s
Jan 24 06:52:20.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:22.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.31971007s
Jan 24 06:52:22.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:24.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.319477386s
Jan 24 06:52:24.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:26.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.320132662s
Jan 24 06:52:26.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:28.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.320198415s
Jan 24 06:52:28.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:30.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.320008518s
Jan 24 06:52:30.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:32.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.319924004s
Jan 24 06:52:32.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:34.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.319675132s
Jan 24 06:52:34.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:36.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.319943551s
Jan 24 06:52:36.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:38.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.319229228s
Jan 24 06:52:38.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:40.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.3199306s
Jan 24 06:52:40.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:42.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.319800058s
Jan 24 06:52:42.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:44.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.319954216s
Jan 24 06:52:44.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:46.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.320026754s
Jan 24 06:52:46.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:48.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.320240162s
Jan 24 06:52:48.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:50.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.319826725s
Jan 24 06:52:50.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:52.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.32003484s
Jan 24 06:52:52.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:54.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.319979874s
Jan 24 06:52:54.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:56.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.319905945s
Jan 24 06:52:56.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:52:58.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.320141468s
Jan 24 06:52:58.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:00.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.320422626s
Jan 24 06:53:00.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:02.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.319981959s
Jan 24 06:53:02.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:04.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.320272459s
Jan 24 06:53:04.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:06.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.319721164s
Jan 24 06:53:06.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:08.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.319555248s
Jan 24 06:53:08.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:10.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.319476605s
Jan 24 06:53:10.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:12.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.319941028s
Jan 24 06:53:12.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:14.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.319528534s
Jan 24 06:53:14.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:16.136: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.321478795s
Jan 24 06:53:16.136: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:18.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.32016298s
Jan 24 06:53:18.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:20.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.319695584s
Jan 24 06:53:20.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:22.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.320645613s
Jan 24 06:53:22.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:24.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.320541088s
Jan 24 06:53:24.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:26.135: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.320205689s
Jan 24 06:53:26.135: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:28.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.319701044s
Jan 24 06:53:28.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:30.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.319796687s
Jan 24 06:53:30.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:32.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.319729751s
Jan 24 06:53:32.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:34.137: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.322402009s
Jan 24 06:53:34.137: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:36.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.319877782s
Jan 24 06:53:36.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:38.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.319307953s
Jan 24 06:53:38.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:40.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.319607356s
Jan 24 06:53:40.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:42.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.319666938s
Jan 24 06:53:42.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:44.134: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.319583045s
Jan 24 06:53:44.134: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:44.293: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.478684335s
Jan 24 06:53:44.293: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:53:44.293: FAIL: Pod did not start running: timed out while waiting for pod port-forwarding-5834/pfpod to be running and ready
Full Stack Trace
k8s.io/kubernetes/test/e2e/kubectl.doTestConnectSendDisconnect({0x75cb648, 0x9}, 0xc000f543c0)
test/e2e/kubectl/portforward.go:217 +0x24c
k8s.io/kubernetes/test/e2e/kubectl.glob..func2.2.2.1()
test/e2e/kubectl/portforward.go:488 +0x29
[AfterEach] [sig-cli] Kubectl Port forwarding
test/e2e/framework/node/init/init.go:32
Jan 24 06:53:44.293: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/24/23 06:53:44.453�[0m
�[1mSTEP:�[0m Collecting events from namespace "port-forwarding-5834". �[38;5;243m01/24/23 06:53:44.454�[0m
�[1mSTEP:�[0m Found 11 events. �[38;5;243m01/24/23 06:53:44.613�[0m
Jan 24 06:53:44.613: INFO: At 2023-01-24 06:48:43 +0000 UTC - event for pfpod: {default-scheduler } Scheduled: Successfully assigned port-forwarding-5834/pfpod to i-036f641c68d5ad392
Jan 24 06:53:44.613: INFO: At 2023-01-24 06:48:44 +0000 UTC - event for pfpod: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7ebf65476e9556472462b6f2039bf39ee994282bb0b2aef3859b1a5d195f9409": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:44.613: INFO: At 2023-01-24 06:48:55 +0000 UTC - event for pfpod: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8881519f7a859d7cc395b3b86f2b5caa13480c64228c1e8a18138bd3ae106f21": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:44.613: INFO: At 2023-01-24 06:49:07 +0000 UTC - event for pfpod: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1351535b7870cfc81ec52b65862183bf33ad38840e0cbec9ac4dea50899b5074": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:44.613: INFO: At 2023-01-24 06:49:22 +0000 UTC - event for pfpod: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d7643902f9dcddf22d026a30377fd0c6ce3f29ce9d564cf77decb10089d2a0f3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:44.613: INFO: At 2023-01-24 06:49:33 +0000 UTC - event for pfpod: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6d64cdd214578a5393abcd7def77547259b9e7df6baec6b793b91c7b12035a9e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:44.613: INFO: At 2023-01-24 06:49:46 +0000 UTC - event for pfpod: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b23a3e5c010a474866df2148bce31a56f6a6cb7da9e7ec9a5c850608829b9cd3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:44.613: INFO: At 2023-01-24 06:49:59 +0000 UTC - event for pfpod: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6540b3f3478dc257beabc94a4dbc9f9584f45e997078229ddb279447927ada37": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:44.613: INFO: At 2023-01-24 06:50:14 +0000 UTC - event for pfpod: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "854103428cbba067596056ac0222a9eb54692c9d48c6b423fe93b09631c2d8b1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:44.613: INFO: At 2023-01-24 06:50:26 +0000 UTC - event for pfpod: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5b7904b66b1a1778bb33b5815c25352d983be9b1e58020dbab22054efd49eef9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:44.613: INFO: At 2023-01-24 06:50:39 +0000 UTC - event for pfpod: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "89a4fc69f39bc37040478b1a809c278a90410005419b536c4fcad9f453f9cd93": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:53:44.772: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 24 06:53:44.772: INFO: pfpod i-036f641c68d5ad392 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:48:43 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:48:43 +0000 UTC ContainersNotReady containers with unready status: [readiness portforwardtester]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:48:43 +0000 UTC ContainersNotReady containers with unready status: [readiness portforwardtester]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:48:43 +0000 UTC }]
Jan 24 06:53:44.772: INFO:
Jan 24 06:53:44.933: INFO: Unable to fetch port-forwarding-5834/pfpod/readiness logs: the server rejected our request for an unknown reason (get pods pfpod)
Jan 24 06:53:45.093: INFO: Unable to fetch port-forwarding-5834/pfpod/portforwardtester logs: the server rejected our request for an unknown reason (get pods pfpod)
Jan 24 06:53:45.254: INFO:
Logging node info for node i-036f641c68d5ad392
Jan 24 06:53:45.414: INFO: Node Info: &Node{ObjectMeta:{i-036f641c68d5ad392 50721840-3696-4207-81a9-8cd089fbb539 35074 0 2023-01-24 06:27:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-036f641c68d5ad392 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-036f641c68d5ad392 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.254 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-5409":"i-036f641c68d5ad392","ebs.csi.aws.com":"i-036f641c68d5ad392"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:53:39 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-24 06:53:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-036f641c68d5ad392,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:53:39 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:53:39 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:53:39 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:53:39 +0000 UTC,LastTransitionTime:2023-01-24 06:27:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.254,},NodeAddress{Type:ExternalIP,Address:15.165.237.223,},NodeAddress{Type:InternalDNS,Address:i-036f641c68d5ad392.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-036f641c68d5ad392.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-165-237-223.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2732c784a9bb7e64a787956c82cf85,SystemUUID:ec2732c7-84a9-bb7e-64a7-87956c82cf85,BootID:bee5ed40-42b1-475e-b72b-16d715de535a,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-5409^2c26217c-9bb3-11ed-bce5-86d6f2bc551f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-5409^2c26217c-9bb3-11ed-bce5-86d6f2bc551f,DevicePath:,},},Config:nil,},}
Jan 24 06:53:45.415: INFO:
Logging kubelet events for node i-036f641c68d5ad392
Jan 24 06:53:45.577: INFO:
Logging pods the kubelet thinks is on node i-036f641c68d5ad392
Jan 24 06:53:45.744: INFO: rs-76gck started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:45.744: INFO: pfpod started at 2023-01-24 06:48:43 +0000 UTC (0+2 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container portforwardtester ready: false, restart count 0
Jan 24 06:53:45.744: INFO: Container readiness ready: false, restart count 0
Jan 24 06:53:45.744: INFO: dns-test-93a0bf56-7463-4d4f-a334-bb3e44707aaa started at 2023-01-24 06:51:48 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container jessie-querier ready: false, restart count 0
Jan 24 06:53:45.744: INFO: Container querier ready: false, restart count 0
Jan 24 06:53:45.744: INFO: Container webserver ready: false, restart count 0
Jan 24 06:53:45.744: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6twph started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:45.744: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-m4sr7 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:45.744: INFO: pod-subpath-test-downwardapi-jp8w started at 2023-01-24 06:53:24 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container test-container-subpath-downwardapi-jp8w ready: false, restart count 0
Jan 24 06:53:45.744: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2mkph started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:45.744: INFO: rs-6s4df started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:45.744: INFO: csi-hostpathplugin-0 started at 2023-01-24 06:44:58 +0000 UTC (0+7 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container csi-attacher ready: true, restart count 0
Jan 24 06:53:45.744: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:53:45.744: INFO: Container csi-resizer ready: true, restart count 0
Jan 24 06:53:45.744: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 24 06:53:45.744: INFO: Container hostpath ready: true, restart count 0
Jan 24 06:53:45.744: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:45.744: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:45.744: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-8sfzb started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:45.744: INFO: rs-zmrj2 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:45.744: INFO: pod-0 started at 2023-01-24 06:53:19 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:45.744: INFO: ebs-csi-node-6z65r started at 2023-01-24 06:27:40 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:45.744: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:45.744: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:45.744: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cpp7h started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:45.744: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6hnbl started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:45.744: INFO: service-headless-btk74 started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container service-headless ready: false, restart count 0
Jan 24 06:53:45.744: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-h5mpm started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:45.744: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-tcp26 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:45.744: INFO: cilium-9ht2g started at 2023-01-24 06:27:40 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:53:45.744: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:53:45.744: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-t9b6f started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:45.744: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-lc7s4 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:45.744: INFO: rand-non-local-gxd9p started at 2023-01-24 06:42:31 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container c ready: false, restart count 0
Jan 24 06:53:45.744: INFO: pod-56c7bad9-2975-4685-aab0-0cfb8b42331c started at 2023-01-24 06:48:56 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container write-pod ready: false, restart count 0
Jan 24 06:53:45.744: INFO: server started at 2023-01-24 06:49:10 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container agnhost-container ready: true, restart count 0
Jan 24 06:53:45.744: INFO: test-ss-0 started at 2023-01-24 06:49:23 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container webserver ready: false, restart count 0
Jan 24 06:53:45.744: INFO: pod-configmaps-5f55ab46-5e11-4c96-87a9-f170ab7204ce started at 2023-01-24 06:49:28 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container agnhost-container ready: false, restart count 0
Jan 24 06:53:45.744: INFO: explicit-nonroot-uid started at 2023-01-24 06:53:27 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:45.744: INFO: Container explicit-nonroot-uid ready: false, restart count 0
Jan 24 06:53:46.349: INFO:
Latency metrics for node i-036f641c68d5ad392
Jan 24 06:53:46.349: INFO:
Logging node info for node i-0697a70b9aa95b7c0
Jan 24 06:53:46.508: INFO: Node Info: &Node{ObjectMeta:{i-0697a70b9aa95b7c0 d0a3c976-eb07-4323-b48b-ec4945de7410 33229 0 2023-01-24 06:27:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0697a70b9aa95b7c0 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0697a70b9aa95b7c0 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.153 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0697a70b9aa95b7c0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:48:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0697a70b9aa95b7c0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:49 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.153,},NodeAddress{Type:ExternalIP,Address:3.38.96.49,},NodeAddress{Type:InternalDNS,Address:i-0697a70b9aa95b7c0.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0697a70b9aa95b7c0.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-38-96-49.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f1e14d470f251c9d042c57478cb8b,SystemUUID:ec2f1e14-d470-f251-c9d0-42c57478cb8b,BootID:578a4c1d-084e-4aeb-92fe-f64671842d82,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:53:46.509: INFO:
Logging kubelet events for node i-0697a70b9aa95b7c0
Jan 24 06:53:46.670: INFO:
Logging pods the kubelet thinks is on node i-0697a70b9aa95b7c0
Jan 24 06:53:46.851: INFO: coredns-autoscaler-7cb5c5b969-vktlf started at 2023-01-24 06:27:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container autoscaler ready: true, restart count 0
Jan 24 06:53:46.851: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-xbpd8 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:53:46.851: INFO: ebs-csi-node-pkrr4 started at 2023-01-24 06:27:36 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:46.851: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:46.851: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:46.851: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-fgc56 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:46.851: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2v7c5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:46.851: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-4sk4h started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:46.851: INFO: hostexec-i-0697a70b9aa95b7c0-n9ccb started at 2023-01-24 06:49:56 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container agnhost-container ready: true, restart count 0
Jan 24 06:53:46.851: INFO: csi-mockplugin-0 started at 2023-01-24 06:48:54 +0000 UTC (0+4 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container busybox ready: false, restart count 0
Jan 24 06:53:46.851: INFO: Container csi-provisioner ready: false, restart count 0
Jan 24 06:53:46.851: INFO: Container driver-registrar ready: false, restart count 0
Jan 24 06:53:46.851: INFO: Container mock ready: false, restart count 0
Jan 24 06:53:46.851: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-68w92 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:46.851: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qphds started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:46.851: INFO: service-headless-xh4pl started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container service-headless ready: false, restart count 0
Jan 24 06:53:46.851: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qtzq5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:46.851: INFO: rs-b9vhw started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container donothing ready: true, restart count 0
Jan 24 06:53:46.851: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2tkvs started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:53:46.851: INFO: cilium-dh7hm started at 2023-01-24 06:27:36 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:53:46.851: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:53:46.851: INFO: coredns-559769c974-6wblg started at 2023-01-24 06:27:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container coredns ready: true, restart count 0
Jan 24 06:53:46.851: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-5pcw5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:46.851: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cdkdr started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:46.851: INFO: rs-w9wjl started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Container donothing ready: true, restart count 0
Jan 24 06:53:46.851: INFO: pod-subpath-test-preprovisionedpv-94gx started at 2023-01-24 06:50:18 +0000 UTC (2+1 container statuses recorded)
Jan 24 06:53:46.851: INFO: Init container init-volume-preprovisionedpv-94gx ready: false, restart count 0
Jan 24 06:53:46.851: INFO: Init container test-init-volume-preprovisionedpv-94gx ready: false, restart count 0
Jan 24 06:53:46.851: INFO: Container test-container-subpath-preprovisionedpv-94gx ready: false, restart count 0
Jan 24 06:53:47.525: INFO:
Latency metrics for node i-0697a70b9aa95b7c0
Jan 24 06:53:47.525: INFO:
Logging node info for node i-0b0b3ac061164cea4
Jan 24 06:53:47.685: INFO: Node Info: &Node{ObjectMeta:{i-0b0b3ac061164cea4 ac00aaf7-4427-49f6-b849-9c812ae80069 34896 0 2023-01-24 06:25:23 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b0b3ac061164cea4 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.51.48 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b0b3ac061164cea4"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-24 06:25:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-24 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:26:22 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:26:22 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-24 06:26:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-24 06:53:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0b0b3ac061164cea4,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892297728 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787440128 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:53:16 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:53:16 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:53:16 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:53:16 +0000 UTC,LastTransitionTime:2023-01-24 06:26:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.48,},NodeAddress{Type:ExternalIP,Address:13.125.222.159,},NodeAddress{Type:InternalDNS,Address:i-0b0b3ac061164cea4.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b0b3ac061164cea4.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-125-222-159.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255807a7dec89d2cc9cfee6d481c20,SystemUUID:ec255807-a7de-c89d-2cc9-cfee6d481c20,BootID:4153aea0-5d07-4f37-a966-c56ef660acfc,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.1],SizeBytes:135178704,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.1],SizeBytes:124995897,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:63920476,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.1],SizeBytes:57661752,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:44339030,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42841305,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4965794,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:53:47.686: INFO:
Logging kubelet events for node i-0b0b3ac061164cea4
Jan 24 06:53:47.847: INFO:
Logging pods the kubelet thinks is on node i-0b0b3ac061164cea4
Jan 24 06:53:48.015: INFO: etcd-manager-events-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (10+1 container statuses recorded)
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Container etcd-manager ready: true, restart count 0
Jan 24 06:53:48.015: INFO: etcd-manager-main-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (10+1 container statuses recorded)
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Container etcd-manager ready: true, restart count 0
Jan 24 06:53:48.015: INFO: kube-controller-manager-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:48.015: INFO: Container kube-controller-manager ready: true, restart count 4
Jan 24 06:53:48.015: INFO: kube-scheduler-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:48.015: INFO: Container kube-scheduler ready: true, restart count 0
Jan 24 06:53:48.015: INFO: cilium-w225f started at 2023-01-24 06:26:07 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:48.015: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:53:48.015: INFO: cilium-operator-5dd44dc49f-rn9bl started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:48.015: INFO: Container cilium-operator ready: true, restart count 0
Jan 24 06:53:48.015: INFO: kube-apiserver-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+2 container statuses recorded)
Jan 24 06:53:48.015: INFO: Container healthcheck ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Container kube-apiserver ready: true, restart count 3
Jan 24 06:53:48.015: INFO: aws-cloud-controller-manager-4n9pv started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:48.015: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 24 06:53:48.015: INFO: ebs-csi-node-q4jsj started at 2023-01-24 06:26:07 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:48.015: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:48.015: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:48.016: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:48.016: INFO: kops-controller-vfj77 started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:48.016: INFO: Container kops-controller ready: true, restart count 0
Jan 24 06:53:48.016: INFO: dns-controller-69987775c6-vzqc8 started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:48.016: INFO: Container dns-controller ready: true, restart count 0
Jan 24 06:53:48.016: INFO: ebs-csi-controller-5bd98b456f-kswjq started at 2023-01-24 06:26:07 +0000 UTC (0+5 container statuses recorded)
Jan 24 06:53:48.016: INFO: Container csi-attacher ready: true, restart count 0
Jan 24 06:53:48.016: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:53:48.016: INFO: Container csi-resizer ready: true, restart count 0
Jan 24 06:53:48.016: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:48.016: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:48.550: INFO:
Latency metrics for node i-0b0b3ac061164cea4
Jan 24 06:53:48.550: INFO:
Logging node info for node i-0e0a289a9d5c333a0
Jan 24 06:53:48.710: INFO: Node Info: &Node{ObjectMeta:{i-0e0a289a9d5c333a0 9194794b-d78a-4a30-bbec-b76df63f68f1 33830 0 2023-01-24 06:27:40 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0e0a289a9d5c333a0 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0e0a289a9d5c333a0 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.62 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1968":"i-0e0a289a9d5c333a0","ebs.csi.aws.com":"i-0e0a289a9d5c333a0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:49:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0e0a289a9d5c333a0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.62,},NodeAddress{Type:ExternalIP,Address:54.180.158.26,},NodeAddress{Type:InternalDNS,Address:i-0e0a289a9d5c333a0.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0e0a289a9d5c333a0.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-180-158-26.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2281a7b00c77d27e8055a22bf9facf,SystemUUID:ec2281a7-b00c-77d2-7e80-55a22bf9facf,BootID:be2e68ba-6839-4b53-847f-22f29f0a7360,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1968^3d6e646f-9bb3-11ed-bc6b-56b051ebca30],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:53:48.710: INFO:
Logging kubelet events for node i-0e0a289a9d5c333a0
Jan 24 06:53:48.878: INFO:
Logging pods the kubelet thinks is on node i-0e0a289a9d5c333a0
Jan 24 06:53:49.047: INFO: pod-subpath-test-preprovisionedpv-qftm started at 2023-01-24 06:53:33 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Init container init-volume-preprovisionedpv-qftm ready: false, restart count 0
Jan 24 06:53:49.048: INFO: Container test-container-subpath-preprovisionedpv-qftm ready: false, restart count 0
Jan 24 06:53:49.048: INFO: csi-mockplugin-attacher-0 started at 2023-01-24 06:48:19 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container csi-attacher ready: false, restart count 0
Jan 24 06:53:49.048: INFO: test-ss-0 started at 2023-01-24 06:49:10 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container webserver ready: false, restart count 0
Jan 24 06:53:49.048: INFO: pod1 started at 2023-01-24 06:50:50 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container agnhost-container ready: false, restart count 0
Jan 24 06:53:49.048: INFO: service-headless-h4z88 started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container service-headless ready: true, restart count 0
Jan 24 06:53:49.048: INFO: dns-test-b3d6c428-d0cf-4d5a-9700-fdfb874b5d91 started at 2023-01-24 06:48:41 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container jessie-querier ready: false, restart count 0
Jan 24 06:53:49.048: INFO: Container querier ready: false, restart count 0
Jan 24 06:53:49.048: INFO: Container webserver ready: false, restart count 0
Jan 24 06:53:49.048: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-9hr5l started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:49.048: INFO: e2e-test-httpd-pod started at 2023-01-24 06:49:16 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container e2e-test-httpd-pod ready: false, restart count 0
Jan 24 06:53:49.048: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-z9lc4 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:49.048: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-7nhf8 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:49.048: INFO: rs-l5fgd started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:49.048: INFO: cilium-cvt86 started at 2023-01-24 06:27:41 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:53:49.048: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:53:49.048: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-224qm started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:49.048: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-zv4q6 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:49.048: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-tbbms started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:49.048: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-w7m2k started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:53:49.048: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-867tg started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:49.048: INFO: rs-rsqdt started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container donothing ready: true, restart count 0
Jan 24 06:53:49.048: INFO: pvc-volume-tester-56wjf started at 2023-01-24 06:49:25 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container volume-tester ready: false, restart count 0
Jan 24 06:53:49.048: INFO: ebs-csi-node-v6prc started at 2023-01-24 06:27:41 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:49.048: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:49.048: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:49.048: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-ts8xr started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:49.048: INFO: hostexec-i-0e0a289a9d5c333a0-7tx4h started at 2023-01-24 06:53:15 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container agnhost-container ready: true, restart count 0
Jan 24 06:53:49.048: INFO: rs-znpr8 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:49.048: INFO: tester started at 2023-01-24 06:53:16 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container tester ready: false, restart count 0
Jan 24 06:53:49.048: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-bkjzk started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:49.048: INFO: csi-mockplugin-0 started at 2023-01-24 06:48:18 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:53:49.048: INFO: Container driver-registrar ready: true, restart count 0
Jan 24 06:53:49.048: INFO: Container mock ready: true, restart count 0
Jan 24 06:53:49.048: INFO: ss-0 started at 2023-01-24 06:53:26 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container webserver ready: false, restart count 0
Jan 24 06:53:49.048: INFO: startup-e83c2b2e-e8d4-4137-b286-5a3397f52961 started at 2023-01-24 06:49:02 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container busybox ready: false, restart count 0
Jan 24 06:53:49.048: INFO: pod-1 started at 2023-01-24 06:53:19 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:49.048: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:49.636: INFO:
Latency metrics for node i-0e0a289a9d5c333a0
Jan 24 06:53:49.636: INFO:
Logging node info for node i-0f5ee4c33b6799e76
Jan 24 06:53:49.795: INFO: Node Info: &Node{ObjectMeta:{i-0f5ee4c33b6799e76 34c5957d-82ce-41f2-824d-90a78fa8bbba 33946 0 2023-01-24 06:27:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0f5ee4c33b6799e76 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0f5ee4c33b6799e76 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.57.132 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0f5ee4c33b6799e76"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:49:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0f5ee4c33b6799e76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.57.132,},NodeAddress{Type:ExternalIP,Address:3.36.124.131,},NodeAddress{Type:InternalDNS,Address:i-0f5ee4c33b6799e76.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f5ee4c33b6799e76.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-36-124-131.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bcf8e071d2b94271b14b41c978c,SystemUUID:ec2f5bcf-8e07-1d2b-9427-1b14b41c978c,BootID:400ceb00-7274-4349-89f8-a3cb13aed9c2,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:53:49.796: INFO:
Logging kubelet events for node i-0f5ee4c33b6799e76
Jan 24 06:53:49.957: INFO:
Logging pods the kubelet thinks is on node i-0f5ee4c33b6799e76
Jan 24 06:53:50.122: INFO: pod-failure-failjob-s2mc2 started at 2023-01-24 06:49:00 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container c ready: false, restart count 0
Jan 24 06:53:50.122: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qmz8q started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:50.122: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cfnfh started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:50.122: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-xl6bg started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:50.122: INFO: rs-g6rhr started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container donothing ready: true, restart count 0
Jan 24 06:53:50.122: INFO: rs-8b8n9 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container donothing ready: false, restart count 0
Jan 24 06:53:50.122: INFO: cilium-ql489 started at 2023-01-24 06:27:40 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:53:50.122: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:53:50.122: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-hwktw started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:50.122: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-7lw99 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:50.122: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-w9729 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:50.122: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-lvvw2 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:50.122: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-wjt45 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:50.122: INFO: coredns-559769c974-5gltc started at 2023-01-24 06:28:16 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container coredns ready: true, restart count 0
Jan 24 06:53:50.122: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-r5vgx started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:50.122: INFO: ebs-csi-node-vjhdb started at 2023-01-24 06:27:40 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:53:50.122: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:53:50.122: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:53:50.122: INFO: pod-failure-failjob-s22zv started at 2023-01-24 06:49:00 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container c ready: false, restart count 0
Jan 24 06:53:50.122: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6lkqq started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:53:50.122: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:53:50.706: INFO:
Latency metrics for node i-0f5ee4c33b6799e76
[DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "port-forwarding-5834" for this suite. �[38;5;243m01/24/23 06:53:50.706�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sclient\sKubectl\slabel\sshould\supdate\sthe\slabel\son\sa\sresource\s\s\[Conformance\]$'
test/e2e/kubectl/kubectl.go:1498 k8s.io/kubernetes/test/e2e/kubectl.glob..func1.21.1() test/e2e/kubectl/kubectl.go:1498 +0x205from junit_01.xml
[BeforeEach] [sig-cli] Kubectl client set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/24/23 06:43:26.614�[0m Jan 24 06:43:26.615: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename kubectl �[38;5;243m01/24/23 06:43:26.615�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/24/23 06:43:27.097�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/24/23 06:43:27.416�[0m [BeforeEach] [sig-cli] Kubectl client test/e2e/framework/metrics/init/init.go:31 [BeforeEach] [sig-cli] Kubectl client test/e2e/kubectl/kubectl.go:274 [BeforeEach] Kubectl label test/e2e/kubectl/kubectl.go:1494 �[1mSTEP:�[0m creating the pod �[38;5;243m01/24/23 06:43:27.734�[0m Jan 24 06:43:27.735: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/19598541-9baf-11ed-b10d-8a774b882db2/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-3256 create -f -' Jan 24 06:43:29.868: INFO: stderr: "" Jan 24 06:43:29.868: INFO: stdout: "pod/pause created\n" Jan 24 06:43:29.868: INFO: Waiting up to 5m0s for 1 pods to be running and ready: [pause] Jan 24 06:43:29.869: INFO: Waiting up to 5m0s for pod "pause" in namespace "kubectl-3256" to be "running and ready" Jan 24 06:43:30.029: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 160.191625ms Jan 24 06:43:30.029: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:43:32.191: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2.322245342s Jan 24 06:43:32.191: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:43:34.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4.321431156s Jan 24 06:43:34.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:43:36.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 6.32165554s Jan 24 06:43:36.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:43:38.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 8.320625385s Jan 24 06:43:38.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:43:40.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 10.321576076s Jan 24 06:43:40.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:43:42.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 12.320926141s Jan 24 06:43:42.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:43:44.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 14.321380945s Jan 24 06:43:44.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:43:46.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 16.321527062s Jan 24 06:43:46.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:43:48.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 18.321149758s Jan 24 06:43:48.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:43:50.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 20.321035751s Jan 24 06:43:50.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:43:52.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 22.32107342s Jan 24 06:43:52.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:43:54.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 24.321272982s Jan 24 06:43:54.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:43:56.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 26.321254909s Jan 24 06:43:56.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:43:58.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 28.320224954s Jan 24 06:43:58.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:00.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 30.321265412s Jan 24 06:44:00.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:02.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 32.320337776s Jan 24 06:44:02.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:04.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 34.321568121s Jan 24 06:44:04.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:06.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 36.321122461s Jan 24 06:44:06.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:08.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 38.32122886s Jan 24 06:44:08.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:10.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 40.321184087s Jan 24 06:44:10.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:12.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 42.321139669s Jan 24 06:44:12.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:14.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 44.320380988s Jan 24 06:44:14.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:16.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 46.321414928s Jan 24 06:44:16.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:18.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 48.321573617s Jan 24 06:44:18.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:20.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 50.320492736s Jan 24 06:44:20.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:22.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 52.321371216s Jan 24 06:44:22.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:24.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 54.321070227s Jan 24 06:44:24.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:26.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 56.321382306s Jan 24 06:44:26.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:28.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 58.320301017s Jan 24 06:44:28.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:30.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.321224946s Jan 24 06:44:30.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:32.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.321090631s Jan 24 06:44:32.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:34.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.321432663s Jan 24 06:44:34.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:36.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.320529089s Jan 24 06:44:36.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:38.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.321399127s Jan 24 06:44:38.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:40.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.320977481s Jan 24 06:44:40.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:42.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.321100658s Jan 24 06:44:42.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:44.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.321479939s Jan 24 06:44:44.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:46.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.321092433s Jan 24 06:44:46.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:48.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.321354802s Jan 24 06:44:48.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:50.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.320642743s Jan 24 06:44:50.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:52.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.321312301s Jan 24 06:44:52.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:54.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.321403841s Jan 24 06:44:54.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:56.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.321293898s Jan 24 06:44:56.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:44:58.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.321153079s Jan 24 06:44:58.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:00.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.321357608s Jan 24 06:45:00.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:02.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.321313144s Jan 24 06:45:02.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:04.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.321174189s Jan 24 06:45:04.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:06.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.321588923s Jan 24 06:45:06.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:08.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.320863798s Jan 24 06:45:08.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:10.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.321276525s Jan 24 06:45:10.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:12.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.321316437s Jan 24 06:45:12.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:14.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.321228674s Jan 24 06:45:14.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:16.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.321135868s Jan 24 06:45:16.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:18.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.321452173s Jan 24 06:45:18.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:20.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.321609764s Jan 24 06:45:20.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:22.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.321360423s Jan 24 06:45:22.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:24.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.321584657s Jan 24 06:45:24.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:26.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.321001381s Jan 24 06:45:26.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:28.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.321181327s Jan 24 06:45:28.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:30.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.321523673s Jan 24 06:45:30.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:32.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.321285615s Jan 24 06:45:32.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:34.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.321095185s Jan 24 06:45:34.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:36.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.321256357s Jan 24 06:45:36.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:38.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.321343571s Jan 24 06:45:38.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:40.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.321618635s Jan 24 06:45:40.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:42.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.321500611s Jan 24 06:45:42.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:44.191: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.322533362s Jan 24 06:45:44.191: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:46.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.320741867s Jan 24 06:45:46.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:48.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.321596735s Jan 24 06:45:48.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:50.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.321557493s Jan 24 06:45:50.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:52.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.32111485s Jan 24 06:45:52.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:54.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.321303326s Jan 24 06:45:54.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:56.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.321283615s Jan 24 06:45:56.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:45:58.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.320443988s Jan 24 06:45:58.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:00.193: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.324580089s Jan 24 06:46:00.193: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:02.191: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.322186211s Jan 24 06:46:02.191: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:04.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.321513514s Jan 24 06:46:04.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:06.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.320833785s Jan 24 06:46:06.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:08.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.321135004s Jan 24 06:46:08.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:10.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.320697991s Jan 24 06:46:10.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:12.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.321007195s Jan 24 06:46:12.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:14.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.321757222s Jan 24 06:46:14.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:16.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.321274807s Jan 24 06:46:16.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:18.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.321602517s Jan 24 06:46:18.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:20.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.32049545s Jan 24 06:46:20.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:22.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.321003879s Jan 24 06:46:22.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:24.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.321009927s Jan 24 06:46:24.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:26.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.321423646s Jan 24 06:46:26.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:28.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.321528887s Jan 24 06:46:28.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:30.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.321102554s Jan 24 06:46:30.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:32.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.321359629s Jan 24 06:46:32.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:34.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.321257206s Jan 24 06:46:34.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:36.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.321112546s Jan 24 06:46:36.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:38.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.321329418s Jan 24 06:46:38.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:40.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.321072827s Jan 24 06:46:40.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:42.191: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.322310121s Jan 24 06:46:42.191: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:44.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.321159037s Jan 24 06:46:44.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:46.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.321343723s Jan 24 06:46:46.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:48.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.320349991s Jan 24 06:46:48.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:50.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.321288341s Jan 24 06:46:50.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:52.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.321034575s Jan 24 06:46:52.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:54.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.321501681s Jan 24 06:46:54.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:56.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.321172083s Jan 24 06:46:56.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:46:58.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.320725527s Jan 24 06:46:58.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:00.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.321144064s Jan 24 06:47:00.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:02.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.321480656s Jan 24 06:47:02.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:04.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.321333989s Jan 24 06:47:04.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:06.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.321364203s Jan 24 06:47:06.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:08.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.321421478s Jan 24 06:47:08.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:10.196: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.327656377s Jan 24 06:47:10.196: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:12.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.321207215s Jan 24 06:47:12.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:14.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.321307929s Jan 24 06:47:14.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:16.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.321519241s Jan 24 06:47:16.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:18.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.320623965s Jan 24 06:47:18.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:20.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.320770853s Jan 24 06:47:20.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:22.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.321302384s Jan 24 06:47:22.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:24.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.321494815s Jan 24 06:47:24.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:26.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.321458358s Jan 24 06:47:26.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:28.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.321362074s Jan 24 06:47:28.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:30.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.321318386s Jan 24 06:47:30.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:32.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.321352616s Jan 24 06:47:32.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:34.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.321436558s Jan 24 06:47:34.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:36.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.321460548s Jan 24 06:47:36.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:38.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.320361052s Jan 24 06:47:38.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:40.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.321208184s Jan 24 06:47:40.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:42.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.321349948s Jan 24 06:47:42.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:44.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.321034795s Jan 24 06:47:44.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:46.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.321305128s Jan 24 06:47:46.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:48.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.320869723s Jan 24 06:47:48.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:50.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.321228523s Jan 24 06:47:50.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:52.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.32150623s Jan 24 06:47:52.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:54.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.321468594s Jan 24 06:47:54.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:56.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.321421051s Jan 24 06:47:56.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:47:58.193: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.324614262s Jan 24 06:47:58.193: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:00.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.321140206s Jan 24 06:48:00.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:02.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.320625091s Jan 24 06:48:02.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:04.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.32143203s Jan 24 06:48:04.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:06.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.321282018s Jan 24 06:48:06.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:08.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.32145043s Jan 24 06:48:08.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:10.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.321160772s Jan 24 06:48:10.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:12.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.321600487s Jan 24 06:48:12.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:14.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.321105706s Jan 24 06:48:14.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:16.190: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.321284802s Jan 24 06:48:16.190: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:18.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.320912488s Jan 24 06:48:18.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:20.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.320753374s Jan 24 06:48:20.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:22.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.320957333s Jan 24 06:48:22.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:24.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.320238165s Jan 24 06:48:24.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:26.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.320750533s Jan 24 06:48:26.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:28.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.320611865s Jan 24 06:48:28.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:30.189: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.320657319s Jan 24 06:48:30.189: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:30.349: INFO: Pod "pause": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.480806772s Jan 24 06:48:30.349: INFO: Error evaluating pod condition running and ready: want pod 'pause' on 'i-0e0a289a9d5c333a0' to be 'Running' but was 'Pending' Jan 24 06:48:30.349: INFO: Pod pause failed to be running and ready. Jan 24 06:48:30.349: INFO: Wanted all 1 pods to be running and ready. Result: false. Pods: [pause] Jan 24 06:48:30.350: FAIL: Expected <bool>: false to equal <bool>: true Full Stack Trace k8s.io/kubernetes/test/e2e/kubectl.glob..func1.21.1() test/e2e/kubectl/kubectl.go:1498 +0x205 [AfterEach] Kubectl label test/e2e/kubectl/kubectl.go:1500 �[1mSTEP:�[0m using delete to clean up resources �[38;5;243m01/24/23 06:48:30.35�[0m Jan 24 06:48:30.350: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/19598541-9baf-11ed-b10d-8a774b882db2/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-3256 delete --grace-period=0 --force -f -' Jan 24 06:48:31.113: INFO: stderr: "Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.\n" Jan 24 06:48:31.113: INFO: stdout: "pod \"pause\" force deleted\n" Jan 24 06:48:31.113: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/19598541-9baf-11ed-b10d-8a774b882db2/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-3256 get rc,svc -l name=pause --no-headers' Jan 24 06:48:31.860: INFO: stderr: "No resources found in kubectl-3256 namespace.\n" Jan 24 06:48:31.860: INFO: stdout: "" Jan 24 06:48:31.860: INFO: Running '/home/prow/go/src/k8s.io/kops/_rundir/19598541-9baf-11ed-b10d-8a774b882db2/kubectl --server=https://api.e2e-e2e-kops-grid-cilium-eni-u2004-k26.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=kubectl-3256 get pods -l name=pause -o go-template={{ range .items }}{{ if not .metadata.deletionTimestamp }}{{ .metadata.name }}{{ "\n" }}{{ end }}{{ end }}' Jan 24 06:48:32.458: INFO: stderr: "" Jan 24 06:48:32.458: INFO: stdout: "" [AfterEach] [sig-cli] Kubectl client test/e2e/framework/node/init/init.go:32 Jan 24 06:48:32.458: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-cli] Kubectl client test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-cli] Kubectl client dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/24/23 06:48:32.619�[0m �[1mSTEP:�[0m Collecting events from namespace "kubectl-3256". �[38;5;243m01/24/23 06:48:32.619�[0m �[1mSTEP:�[0m Found 11 events. �[38;5;243m01/24/23 06:48:32.779�[0m Jan 24 06:48:32.780: INFO: At 2023-01-24 06:43:29 +0000 UTC - event for pause: {default-scheduler } Scheduled: Successfully assigned kubectl-3256/pause to i-0e0a289a9d5c333a0 Jan 24 06:48:32.780: INFO: At 2023-01-24 06:43:30 +0000 UTC - event for pause: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "042bace13ba51dfd953be18ebc182ac3420e260070dd61e2641bea46450905a7": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:48:32.780: INFO: At 2023-01-24 06:43:42 +0000 UTC - event for pause: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b1fe55cf6d7b6dfa577883822ad5e78b89bf074a14fec3e181b081ebf96a7620": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:48:32.780: INFO: At 2023-01-24 06:43:56 +0000 UTC - event for pause: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d2da4a3accdce8cfd0c42c881e46721b1f1de83002cb8e0107b4d52996175093": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:48:32.780: INFO: At 2023-01-24 06:44:07 +0000 UTC - event for pause: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "29c7ee2d48e9c489e91510f37b0cd2397797384a8b32b6ecdca1c0b09236efa8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:48:32.780: INFO: At 2023-01-24 06:44:22 +0000 UTC - event for pause: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a8714c8785769542d5107dae8f605ace1c2cae65f2ab4a49811f33398837237b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:48:32.780: INFO: At 2023-01-24 06:44:36 +0000 UTC - event for pause: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9a4de290ac1d78926910bcc4da34fd64263aa76e7dade8d66c77e11dd72f6d09": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:48:32.780: INFO: At 2023-01-24 06:44:50 +0000 UTC - event for pause: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "03be889c8034fb8a2086acda638d39612031ad8f417598ba8aa807d5d04bf552": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:48:32.780: INFO: At 2023-01-24 06:45:05 +0000 UTC - event for pause: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bf3a458ca025d0faa39d2c1f9ef469205b0a33b942a4b9332b41490baa1d3c9b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:48:32.780: INFO: At 2023-01-24 06:45:17 +0000 UTC - event for pause: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "94b819e4a1569a4ac39b557846c8964704f7f192d00e8a89b2c9e8485474353b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:48:32.780: INFO: At 2023-01-24 06:45:30 +0000 UTC - event for pause: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "719ae7fe2f39d1b5b01cd780e4a7ebd744715015e0f8ec8777ada9baa99c5470": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:48:32.939: INFO: POD NODE PHASE GRACE CONDITIONS Jan 24 06:48:32.939: INFO: Jan 24 06:48:33.100: INFO: Logging node info for node i-036f641c68d5ad392 Jan 24 06:48:33.260: INFO: Node Info: &Node{ObjectMeta:{i-036f641c68d5ad392 50721840-3696-4207-81a9-8cd089fbb539 31690 0 2023-01-24 06:27:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-036f641c68d5ad392 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-036f641c68d5ad392 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.254 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-8398":"i-036f641c68d5ad392","ebs.csi.aws.com":"i-036f641c68d5ad392"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:44:10 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-24 06:44:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-036f641c68d5ad392,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:18 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:18 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:18 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:44:18 +0000 UTC,LastTransitionTime:2023-01-24 06:27:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.254,},NodeAddress{Type:ExternalIP,Address:15.165.237.223,},NodeAddress{Type:InternalDNS,Address:i-036f641c68d5ad392.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-036f641c68d5ad392.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-165-237-223.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2732c784a9bb7e64a787956c82cf85,SystemUUID:ec2732c7-84a9-bb7e-64a7-87956c82cf85,BootID:bee5ed40-42b1-475e-b72b-16d715de535a,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-8398^8152cf6c-9bb2-11ed-ba50-629dc9cbc81c kubernetes.io/csi/ebs.csi.aws.com^vol-0269089ae9a4173b7],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0269089ae9a4173b7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-8398^8152cf6c-9bb2-11ed-ba50-629dc9cbc81c,DevicePath:,},},Config:nil,},} Jan 24 06:48:33.261: INFO: Logging kubelet events for node i-036f641c68d5ad392 Jan 24 06:48:33.423: INFO: Logging pods the kubelet thinks is on node i-036f641c68d5ad392 Jan 24 06:48:33.597: INFO: ebs-csi-node-6z65r started at 2023-01-24 06:27:40 +0000 UTC (0+3 container statuses recorded) Jan 24 06:48:33.597: INFO: Container ebs-plugin ready: true, restart count 0 Jan 24 06:48:33.597: INFO: Container liveness-probe ready: true, restart count 0 Jan 24 06:48:33.597: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 24 06:48:33.597: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cpp7h started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:33.597: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6hnbl started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:33.597: INFO: pod-subpath-test-dynamicpv-s6jk started at 2023-01-24 06:44:10 +0000 UTC (1+2 container statuses recorded) Jan 24 06:48:33.597: INFO: Init container test-init-subpath-dynamicpv-s6jk ready: false, restart count 0 Jan 24 06:48:33.597: INFO: Container test-container-subpath-dynamicpv-s6jk ready: false, restart count 0 Jan 24 06:48:33.597: INFO: Container test-container-volume-dynamicpv-s6jk ready: false, restart count 0 Jan 24 06:48:33.597: INFO: service-headless-btk74 started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container service-headless ready: false, restart count 0 Jan 24 06:48:33.597: INFO: pod-da9584f9-0d79-4167-ad37-ca206c3ac911 started at 2023-01-24 06:43:12 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container write-pod ready: false, restart count 0 Jan 24 06:48:33.597: INFO: csi-mockplugin-resizer-0 started at 2023-01-24 06:47:12 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container csi-resizer ready: false, restart count 0 Jan 24 06:48:33.597: INFO: pod-subpath-test-preprovisionedpv-l2p7 started at 2023-01-24 06:48:02 +0000 UTC (1+2 container statuses recorded) Jan 24 06:48:33.597: INFO: Init container test-init-subpath-preprovisionedpv-l2p7 ready: false, restart count 0 Jan 24 06:48:33.597: INFO: Container test-container-subpath-preprovisionedpv-l2p7 ready: false, restart count 0 Jan 24 06:48:33.597: INFO: Container test-container-volume-preprovisionedpv-l2p7 ready: false, restart count 0 Jan 24 06:48:33.597: INFO: pod-handle-http-request started at 2023-01-24 06:48:15 +0000 UTC (0+2 container statuses recorded) Jan 24 06:48:33.597: INFO: Container container-handle-http-request ready: false, restart count 0 Jan 24 06:48:33.597: INFO: Container container-handle-https-request ready: false, restart count 0 Jan 24 06:48:33.597: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-h5mpm started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:33.597: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-tcp26 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:33.597: INFO: cilium-9ht2g started at 2023-01-24 06:27:40 +0000 UTC (1+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 24 06:48:33.597: INFO: Container cilium-agent ready: true, restart count 0 Jan 24 06:48:33.597: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-t9b6f started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:33.597: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-lc7s4 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:33.597: INFO: rand-non-local-gxd9p started at 2023-01-24 06:42:31 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container c ready: false, restart count 0 Jan 24 06:48:33.597: INFO: netserver-0 started at 2023-01-24 06:43:38 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container webserver ready: true, restart count 0 Jan 24 06:48:33.597: INFO: rs-76gck started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container donothing ready: false, restart count 0 Jan 24 06:48:33.597: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6twph started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:33.597: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-m4sr7 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:33.597: INFO: csi-mockplugin-0 started at 2023-01-24 06:47:12 +0000 UTC (0+3 container statuses recorded) Jan 24 06:48:33.597: INFO: Container csi-provisioner ready: false, restart count 0 Jan 24 06:48:33.597: INFO: Container driver-registrar ready: false, restart count 0 Jan 24 06:48:33.597: INFO: Container mock ready: false, restart count 0 Jan 24 06:48:33.597: INFO: hostexec-i-036f641c68d5ad392-g49z8 started at 2023-01-24 06:47:49 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container agnhost-container ready: true, restart count 0 Jan 24 06:48:33.597: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2mkph started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:33.597: INFO: rs-6s4df started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container donothing ready: false, restart count 0 Jan 24 06:48:33.597: INFO: csi-hostpathplugin-0 started at 2023-01-24 06:44:58 +0000 UTC (0+7 container statuses recorded) Jan 24 06:48:33.597: INFO: Container csi-attacher ready: false, restart count 0 Jan 24 06:48:33.597: INFO: Container csi-provisioner ready: false, restart count 0 Jan 24 06:48:33.597: INFO: Container csi-resizer ready: false, restart count 0 Jan 24 06:48:33.597: INFO: Container csi-snapshotter ready: false, restart count 0 Jan 24 06:48:33.597: INFO: Container hostpath ready: false, restart count 0 Jan 24 06:48:33.597: INFO: Container liveness-probe ready: false, restart count 0 Jan 24 06:48:33.597: INFO: Container node-driver-registrar ready: false, restart count 0 Jan 24 06:48:33.597: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-8sfzb started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0 Jan 24 06:48:33.597: INFO: rs-zmrj2 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container donothing ready: false, restart count 0 Jan 24 06:48:33.597: INFO: csi-hostpathplugin-0 started at 2023-01-24 06:42:31 +0000 UTC (0+7 container statuses recorded) Jan 24 06:48:33.597: INFO: Container csi-attacher ready: true, restart count 0 Jan 24 06:48:33.597: INFO: Container csi-provisioner ready: true, restart count 0 Jan 24 06:48:33.597: INFO: Container csi-resizer ready: true, restart count 0 Jan 24 06:48:33.597: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 24 06:48:33.597: INFO: Container hostpath ready: true, restart count 0 Jan 24 06:48:33.597: INFO: Container liveness-probe ready: true, restart count 0 Jan 24 06:48:33.597: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 24 06:48:33.597: INFO: netserver-0 started at 2023-01-24 06:42:43 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:33.597: INFO: Container webserver ready: true, restart count 0 Jan 24 06:48:34.222: INFO: Latency metrics for node i-036f641c68d5ad392 Jan 24 06:48:34.222: INFO: Logging node info for node i-0697a70b9aa95b7c0 Jan 24 06:48:34.382: INFO: Node Info: &Node{ObjectMeta:{i-0697a70b9aa95b7c0 d0a3c976-eb07-4323-b48b-ec4945de7410 31075 0 2023-01-24 06:27:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0697a70b9aa95b7c0 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0697a70b9aa95b7c0 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.153 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0697a70b9aa95b7c0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:43:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0697a70b9aa95b7c0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:43:35 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:43:35 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:43:35 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:43:35 +0000 UTC,LastTransitionTime:2023-01-24 06:27:49 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.153,},NodeAddress{Type:ExternalIP,Address:3.38.96.49,},NodeAddress{Type:InternalDNS,Address:i-0697a70b9aa95b7c0.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0697a70b9aa95b7c0.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-38-96-49.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f1e14d470f251c9d042c57478cb8b,SystemUUID:ec2f1e14-d470-f251-c9d0-42c57478cb8b,BootID:578a4c1d-084e-4aeb-92fe-f64671842d82,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 24 06:48:34.382: INFO: Logging kubelet events for node i-0697a70b9aa95b7c0 Jan 24 06:48:34.545: INFO: Logging pods the kubelet thinks is on node i-0697a70b9aa95b7c0 Jan 24 06:48:34.713: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-68w92 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:34.713: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qphds started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:34.713: INFO: webserver-pod started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container agnhost ready: false, restart count 0 Jan 24 06:48:34.713: INFO: hostexec-i-0697a70b9aa95b7c0-g8mhd started at 2023-01-24 06:43:35 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container agnhost-container ready: true, restart count 0 Jan 24 06:48:34.713: INFO: service-headless-xh4pl started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container service-headless ready: false, restart count 0 Jan 24 06:48:34.713: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qtzq5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:34.713: INFO: netserver-1 started at 2023-01-24 06:43:39 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container webserver ready: false, restart count 0 Jan 24 06:48:34.713: INFO: rs-b9vhw started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container donothing ready: false, restart count 0 Jan 24 06:48:34.713: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2tkvs started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0 Jan 24 06:48:34.713: INFO: pod-subpath-test-inlinevolume-qdbt started at 2023-01-24 06:44:17 +0000 UTC (1+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Init container init-volume-inlinevolume-qdbt ready: false, restart count 0 Jan 24 06:48:34.713: INFO: Container test-container-subpath-inlinevolume-qdbt ready: false, restart count 0 Jan 24 06:48:34.713: INFO: netserver-1 started at 2023-01-24 06:42:43 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container webserver ready: true, restart count 0 Jan 24 06:48:34.713: INFO: csi-mockplugin-attacher-0 started at 2023-01-24 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container csi-attacher ready: false, restart count 0 Jan 24 06:48:34.713: INFO: cilium-dh7hm started at 2023-01-24 06:27:36 +0000 UTC (1+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 24 06:48:34.713: INFO: Container cilium-agent ready: true, restart count 0 Jan 24 06:48:34.713: INFO: coredns-559769c974-6wblg started at 2023-01-24 06:27:49 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container coredns ready: true, restart count 0 Jan 24 06:48:34.713: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-5pcw5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:34.713: INFO: exec-volume-test-preprovisionedpv-zb7d started at 2023-01-24 06:43:48 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container exec-container-preprovisionedpv-zb7d ready: false, restart count 0 Jan 24 06:48:34.713: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cdkdr started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:34.713: INFO: rs-w9wjl started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container donothing ready: false, restart count 0 Jan 24 06:48:34.713: INFO: csi-mockplugin-0 started at 2023-01-24 06:44:33 +0000 UTC (0+3 container statuses recorded) Jan 24 06:48:34.713: INFO: Container csi-provisioner ready: false, restart count 0 Jan 24 06:48:34.713: INFO: Container driver-registrar ready: false, restart count 0 Jan 24 06:48:34.713: INFO: Container mock ready: false, restart count 0 Jan 24 06:48:34.713: INFO: csi-mockplugin-resizer-0 started at 2023-01-24 06:44:33 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container csi-resizer ready: false, restart count 0 Jan 24 06:48:34.713: INFO: coredns-autoscaler-7cb5c5b969-vktlf started at 2023-01-24 06:27:49 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container autoscaler ready: true, restart count 0 Jan 24 06:48:34.713: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-xbpd8 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0 Jan 24 06:48:34.713: INFO: ebs-csi-node-pkrr4 started at 2023-01-24 06:27:36 +0000 UTC (0+3 container statuses recorded) Jan 24 06:48:34.713: INFO: Container ebs-plugin ready: true, restart count 0 Jan 24 06:48:34.713: INFO: Container liveness-probe ready: true, restart count 0 Jan 24 06:48:34.713: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 24 06:48:34.713: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-fgc56 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:34.713: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2v7c5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:34.713: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-4sk4h started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:34.713: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:35.365: INFO: Latency metrics for node i-0697a70b9aa95b7c0 Jan 24 06:48:35.365: INFO: Logging node info for node i-0b0b3ac061164cea4 Jan 24 06:48:35.525: INFO: Node Info: &Node{ObjectMeta:{i-0b0b3ac061164cea4 ac00aaf7-4427-49f6-b849-9c812ae80069 33008 0 2023-01-24 06:25:23 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b0b3ac061164cea4 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.51.48 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b0b3ac061164cea4"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-24 06:25:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-24 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:26:22 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:26:22 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-24 06:26:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-24 06:48:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0b0b3ac061164cea4,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892297728 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787440128 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:26:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.48,},NodeAddress{Type:ExternalIP,Address:13.125.222.159,},NodeAddress{Type:InternalDNS,Address:i-0b0b3ac061164cea4.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b0b3ac061164cea4.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-125-222-159.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255807a7dec89d2cc9cfee6d481c20,SystemUUID:ec255807-a7de-c89d-2cc9-cfee6d481c20,BootID:4153aea0-5d07-4f37-a966-c56ef660acfc,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.1],SizeBytes:135178704,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.1],SizeBytes:124995897,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:63920476,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.1],SizeBytes:57661752,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:44339030,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42841305,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4965794,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 24 06:48:35.526: INFO: Logging kubelet events for node i-0b0b3ac061164cea4 Jan 24 06:48:35.689: INFO: Logging pods the kubelet thinks is on node i-0b0b3ac061164cea4 Jan 24 06:48:35.857: INFO: cilium-w225f started at 2023-01-24 06:26:07 +0000 UTC (1+1 container statuses recorded) Jan 24 06:48:35.857: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Container cilium-agent ready: true, restart count 0 Jan 24 06:48:35.857: INFO: cilium-operator-5dd44dc49f-rn9bl started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:35.857: INFO: Container cilium-operator ready: true, restart count 0 Jan 24 06:48:35.857: INFO: etcd-manager-events-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (10+1 container statuses recorded) Jan 24 06:48:35.857: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Container etcd-manager ready: true, restart count 0 Jan 24 06:48:35.857: INFO: etcd-manager-main-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (10+1 container statuses recorded) Jan 24 06:48:35.857: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Container etcd-manager ready: true, restart count 0 Jan 24 06:48:35.857: INFO: kube-controller-manager-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:35.857: INFO: Container kube-controller-manager ready: true, restart count 4 Jan 24 06:48:35.857: INFO: kube-scheduler-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:35.857: INFO: Container kube-scheduler ready: true, restart count 0 Jan 24 06:48:35.857: INFO: dns-controller-69987775c6-vzqc8 started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:35.857: INFO: Container dns-controller ready: true, restart count 0 Jan 24 06:48:35.857: INFO: ebs-csi-controller-5bd98b456f-kswjq started at 2023-01-24 06:26:07 +0000 UTC (0+5 container statuses recorded) Jan 24 06:48:35.857: INFO: Container csi-attacher ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Container csi-provisioner ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Container csi-resizer ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Container ebs-plugin ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Container liveness-probe ready: true, restart count 0 Jan 24 06:48:35.857: INFO: kube-apiserver-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+2 container statuses recorded) Jan 24 06:48:35.857: INFO: Container healthcheck ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Container kube-apiserver ready: true, restart count 3 Jan 24 06:48:35.857: INFO: aws-cloud-controller-manager-4n9pv started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:35.857: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 24 06:48:35.857: INFO: ebs-csi-node-q4jsj started at 2023-01-24 06:26:07 +0000 UTC (0+3 container statuses recorded) Jan 24 06:48:35.857: INFO: Container ebs-plugin ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Container liveness-probe ready: true, restart count 0 Jan 24 06:48:35.857: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 24 06:48:35.857: INFO: kops-controller-vfj77 started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:35.857: INFO: Container kops-controller ready: true, restart count 0 Jan 24 06:48:36.417: INFO: Latency metrics for node i-0b0b3ac061164cea4 Jan 24 06:48:36.417: INFO: Logging node info for node i-0e0a289a9d5c333a0 Jan 24 06:48:36.577: INFO: Node Info: &Node{ObjectMeta:{i-0e0a289a9d5c333a0 9194794b-d78a-4a30-bbec-b76df63f68f1 31709 0 2023-01-24 06:27:40 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0e0a289a9d5c333a0 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0e0a289a9d5c333a0 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.62 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0e0a289a9d5c333a0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:44:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0e0a289a9d5c333a0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:10 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:10 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:10 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:44:10 +0000 UTC,LastTransitionTime:2023-01-24 06:27:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.62,},NodeAddress{Type:ExternalIP,Address:54.180.158.26,},NodeAddress{Type:InternalDNS,Address:i-0e0a289a9d5c333a0.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0e0a289a9d5c333a0.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-180-158-26.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2281a7b00c77d27e8055a22bf9facf,SystemUUID:ec2281a7-b00c-77d2-7e80-55a22bf9facf,BootID:be2e68ba-6839-4b53-847f-22f29f0a7360,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 24 06:48:36.578: INFO: Logging kubelet events for node i-0e0a289a9d5c333a0 Jan 24 06:48:36.740: INFO: Logging pods the kubelet thinks is on node i-0e0a289a9d5c333a0 Jan 24 06:48:36.909: INFO: alpine-nnp-false-2c8bfdf9-3bfb-42e5-835c-fb8faa43651e started at 2023-01-24 06:46:40 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container alpine-nnp-false-2c8bfdf9-3bfb-42e5-835c-fb8faa43651e ready: false, restart count 0 Jan 24 06:48:36.909: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-9hr5l started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:36.909: INFO: netserver-2 started at 2023-01-24 06:43:39 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container webserver ready: true, restart count 0 Jan 24 06:48:36.909: INFO: test-orphan-deployment-7f5969cbc7-b98k5 started at 2023-01-24 06:44:00 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container httpd ready: false, restart count 0 Jan 24 06:48:36.909: INFO: service-headless-h4z88 started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container service-headless ready: false, restart count 0 Jan 24 06:48:36.909: INFO: rs-l5fgd started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container donothing ready: false, restart count 0 Jan 24 06:48:36.909: INFO: netserver-2 started at 2023-01-24 06:42:43 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container webserver ready: true, restart count 0 Jan 24 06:48:36.909: INFO: pod-d2fe4cd9-5425-48c9-8a72-8417570ba681 started at 2023-01-24 06:47:54 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container test-container ready: false, restart count 0 Jan 24 06:48:36.909: INFO: cilium-cvt86 started at 2023-01-24 06:27:41 +0000 UTC (1+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 24 06:48:36.909: INFO: Container cilium-agent ready: true, restart count 0 Jan 24 06:48:36.909: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-224qm started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:36.909: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-z9lc4 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:36.909: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-7nhf8 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:36.909: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-w7m2k started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0 Jan 24 06:48:36.909: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-867tg started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:36.909: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-zv4q6 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:36.909: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-tbbms started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:36.909: INFO: ebs-csi-node-v6prc started at 2023-01-24 06:27:41 +0000 UTC (0+3 container statuses recorded) Jan 24 06:48:36.909: INFO: Container ebs-plugin ready: true, restart count 0 Jan 24 06:48:36.909: INFO: Container liveness-probe ready: true, restart count 0 Jan 24 06:48:36.909: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 24 06:48:36.909: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-ts8xr started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:36.909: INFO: rs-rsqdt started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container donothing ready: true, restart count 0 Jan 24 06:48:36.909: INFO: rc-test-qrc82 started at 2023-01-24 06:48:11 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container rc-test ready: false, restart count 0 Jan 24 06:48:36.909: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-bkjzk started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:36.909: INFO: csi-mockplugin-0 started at 2023-01-24 06:48:18 +0000 UTC (0+3 container statuses recorded) Jan 24 06:48:36.909: INFO: Container csi-provisioner ready: false, restart count 0 Jan 24 06:48:36.909: INFO: Container driver-registrar ready: false, restart count 0 Jan 24 06:48:36.909: INFO: Container mock ready: false, restart count 0 Jan 24 06:48:36.909: INFO: rs-znpr8 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container donothing ready: false, restart count 0 Jan 24 06:48:36.909: INFO: csi-mockplugin-attacher-0 started at 2023-01-24 06:48:19 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:36.909: INFO: Container csi-attacher ready: false, restart count 0 Jan 24 06:48:37.535: INFO: Latency metrics for node i-0e0a289a9d5c333a0 Jan 24 06:48:37.535: INFO: Logging node info for node i-0f5ee4c33b6799e76 Jan 24 06:48:37.695: INFO: Node Info: &Node{ObjectMeta:{i-0f5ee4c33b6799e76 34c5957d-82ce-41f2-824d-90a78fa8bbba 31698 0 2023-01-24 06:27:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0f5ee4c33b6799e76 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0f5ee4c33b6799e76 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.57.132 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7943":"i-0f5ee4c33b6799e76","ebs.csi.aws.com":"i-0f5ee4c33b6799e76"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:44:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0f5ee4c33b6799e76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:19 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:19 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:19 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:44:19 +0000 UTC,LastTransitionTime:2023-01-24 06:27:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.57.132,},NodeAddress{Type:ExternalIP,Address:3.36.124.131,},NodeAddress{Type:InternalDNS,Address:i-0f5ee4c33b6799e76.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f5ee4c33b6799e76.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-36-124-131.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bcf8e071d2b94271b14b41c978c,SystemUUID:ec2f5bcf-8e07-1d2b-9427-1b14b41c978c,BootID:400ceb00-7274-4349-89f8-a3cb13aed9c2,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 24 06:48:37.695: INFO: Logging kubelet events for node i-0f5ee4c33b6799e76 Jan 24 06:48:37.857: INFO: Logging pods the kubelet thinks is on node i-0f5ee4c33b6799e76 Jan 24 06:48:38.028: INFO: hostexec-i-0f5ee4c33b6799e76-g6qhd started at 2023-01-24 06:43:32 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container agnhost-container ready: true, restart count 0 Jan 24 06:48:38.028: INFO: pod-e22bfd9d-fd2c-4c39-bf88-7771a5c4ec25 started at 2023-01-24 06:44:00 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container write-pod ready: false, restart count 0 Jan 24 06:48:38.028: INFO: hostexec-i-0f5ee4c33b6799e76-26jmv started at 2023-01-24 06:43:55 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container agnhost-container ready: true, restart count 0 Jan 24 06:48:38.028: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qmz8q started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:38.028: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-hwktw started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:38.028: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cfnfh started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:38.028: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-xl6bg started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:38.028: INFO: rs-g6rhr started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container donothing ready: false, restart count 0 Jan 24 06:48:38.028: INFO: rs-8b8n9 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container donothing ready: false, restart count 0 Jan 24 06:48:38.028: INFO: pvc-volume-tester-slzmq started at 2023-01-24 06:44:20 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container volume-tester ready: false, restart count 0 Jan 24 06:48:38.028: INFO: cilium-ql489 started at 2023-01-24 06:27:40 +0000 UTC (1+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 24 06:48:38.028: INFO: Container cilium-agent ready: true, restart count 0 Jan 24 06:48:38.028: INFO: pod-ca8a4ceb-b094-43f3-aee3-215db13281b7 started at 2023-01-24 06:43:37 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container write-pod ready: false, restart count 0 Jan 24 06:48:38.028: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-7lw99 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:38.028: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-w9729 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:38.028: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-lvvw2 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:38.028: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-wjt45 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:38.028: INFO: coredns-559769c974-5gltc started at 2023-01-24 06:28:16 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container coredns ready: true, restart count 0 Jan 24 06:48:38.028: INFO: netserver-3 started at 2023-01-24 06:42:43 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container webserver ready: false, restart count 0 Jan 24 06:48:38.028: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-r5vgx started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:38.028: INFO: csi-mockplugin-0 started at 2023-01-24 06:43:09 +0000 UTC (0+3 container statuses recorded) Jan 24 06:48:38.028: INFO: Container csi-provisioner ready: true, restart count 0 Jan 24 06:48:38.028: INFO: Container driver-registrar ready: true, restart count 0 Jan 24 06:48:38.028: INFO: Container mock ready: true, restart count 0 Jan 24 06:48:38.028: INFO: netserver-3 started at 2023-01-24 06:43:39 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container webserver ready: false, restart count 0 Jan 24 06:48:38.028: INFO: ebs-csi-node-vjhdb started at 2023-01-24 06:27:40 +0000 UTC (0+3 container statuses recorded) Jan 24 06:48:38.028: INFO: Container ebs-plugin ready: true, restart count 0 Jan 24 06:48:38.028: INFO: Container liveness-probe ready: true, restart count 0 Jan 24 06:48:38.028: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 24 06:48:38.028: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6lkqq started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:48:38.028: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:48:38.629: INFO: Latency metrics for node i-0f5ee4c33b6799e76 [DeferCleanup (Each)] [sig-cli] Kubectl client tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "kubectl-3256" for this suite. �[38;5;243m01/24/23 06:48:38.629�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sNetworking\sGranular\sChecks\:\sServices\sshould\sbe\sable\sto\shandle\slarge\srequests\:\shttp$'
test/e2e/framework/network/utils.go:866
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc0036b2000, {0x75cb852, 0x9}, 0xc005f03560)
test/e2e/framework/network/utils.go:866 +0x1d0
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc0036b2000, 0x7ff5d9a87758?)
test/e2e/framework/network/utils.go:763 +0x55
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc0036b2000, 0x3e?)
test/e2e/framework/network/utils.go:778 +0x3e
k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc0004085a0, {0x0, 0x0, 0x0?})
test/e2e/framework/network/utils.go:131 +0x125
k8s.io/kubernetes/test/e2e/network.glob..func22.6.18()
test/e2e/network/networking.go:452 +0x36
[BeforeEach] [sig-network] Networking
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/24/23 06:42:41.729�[0m
Jan 24 06:42:41.729: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename nettest �[38;5;243m01/24/23 06:42:41.73�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/24/23 06:42:42.21�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/24/23 06:42:42.527�[0m
[BeforeEach] [sig-network] Networking
test/e2e/framework/metrics/init/init.go:31
[It] should be able to handle large requests: http
test/e2e/network/networking.go:451
�[1mSTEP:�[0m Performing setup for networking test in namespace nettest-2681 �[38;5;243m01/24/23 06:42:42.844�[0m
�[1mSTEP:�[0m creating a selector �[38;5;243m01/24/23 06:42:42.844�[0m
�[1mSTEP:�[0m Creating the service pods in kubernetes �[38;5;243m01/24/23 06:42:42.845�[0m
Jan 24 06:42:42.845: INFO: Waiting up to 10m0s for all (but 0) nodes to be schedulable
Jan 24 06:42:43.817: INFO: Waiting up to 5m0s for pod "netserver-0" in namespace "nettest-2681" to be "running and ready"
Jan 24 06:42:43.977: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 159.318063ms
Jan 24 06:42:43.977: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:42:46.136: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2.319138413s
Jan 24 06:42:46.136: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:42:48.137: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4.319773164s
Jan 24 06:42:48.137: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:42:50.137: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 6.319913441s
Jan 24 06:42:50.137: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:42:52.136: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 8.319055847s
Jan 24 06:42:52.136: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:42:54.136: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 10.318782661s
Jan 24 06:42:54.136: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:42:56.139: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 12.322190353s
Jan 24 06:42:56.139: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:42:58.137: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 14.320265376s
Jan 24 06:42:58.138: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:00.137: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 16.31974425s
Jan 24 06:43:00.137: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:02.136: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 18.319062506s
Jan 24 06:43:02.136: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:04.137: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 20.320016176s
Jan 24 06:43:04.137: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:06.136: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 22.319145707s
Jan 24 06:43:06.136: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:08.136: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 24.318991083s
Jan 24 06:43:08.136: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:10.136: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 26.319008135s
Jan 24 06:43:10.136: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:12.137: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 28.319637393s
Jan 24 06:43:12.137: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:14.137: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 30.319607988s
Jan 24 06:43:14.137: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 24 06:43:16.137: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 32.320273786s
Jan 24 06:43:16.138: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 24 06:43:18.136: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 34.318792575s
Jan 24 06:43:18.136: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 24 06:43:20.137: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 36.319463802s
Jan 24 06:43:20.137: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 24 06:43:22.137: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 38.320246646s
Jan 24 06:43:22.137: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 24 06:43:24.137: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 40.320007172s
Jan 24 06:43:24.137: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 24 06:43:26.137: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=true. Elapsed: 42.319701879s
Jan 24 06:43:26.137: INFO: The phase of Pod netserver-0 is Running (Ready = true)
Jan 24 06:43:26.137: INFO: Pod "netserver-0" satisfied condition "running and ready"
Jan 24 06:43:26.296: INFO: Waiting up to 5m0s for pod "netserver-1" in namespace "nettest-2681" to be "running and ready"
Jan 24 06:43:26.455: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=true. Elapsed: 159.007834ms
Jan 24 06:43:26.455: INFO: The phase of Pod netserver-1 is Running (Ready = true)
Jan 24 06:43:26.455: INFO: Pod "netserver-1" satisfied condition "running and ready"
Jan 24 06:43:26.614: INFO: Waiting up to 5m0s for pod "netserver-2" in namespace "nettest-2681" to be "running and ready"
Jan 24 06:43:26.774: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 159.517877ms
Jan 24 06:43:26.774: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Jan 24 06:43:28.935: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 2.320626207s
Jan 24 06:43:28.935: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Jan 24 06:43:30.934: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 4.319621173s
Jan 24 06:43:30.934: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Jan 24 06:43:32.934: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 6.319535446s
Jan 24 06:43:32.934: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Jan 24 06:43:34.935: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=true. Elapsed: 8.320043879s
Jan 24 06:43:34.935: INFO: The phase of Pod netserver-2 is Running (Ready = true)
Jan 24 06:43:34.935: INFO: Pod "netserver-2" satisfied condition "running and ready"
Jan 24 06:43:35.094: INFO: Waiting up to 5m0s for pod "netserver-3" in namespace "nettest-2681" to be "running and ready"
Jan 24 06:43:35.253: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 159.570037ms
Jan 24 06:43:35.253: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:37.416: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2.322207414s
Jan 24 06:43:37.416: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:39.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4.320215795s
Jan 24 06:43:39.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:41.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 6.319605523s
Jan 24 06:43:41.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:43.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 8.31975247s
Jan 24 06:43:43.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:45.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 10.319749587s
Jan 24 06:43:45.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:47.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 12.319342474s
Jan 24 06:43:47.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:49.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 14.320132747s
Jan 24 06:43:49.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:51.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 16.320325168s
Jan 24 06:43:51.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:53.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 18.319879087s
Jan 24 06:43:53.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:55.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 20.320089684s
Jan 24 06:43:55.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:57.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 22.319439443s
Jan 24 06:43:57.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:59.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 24.319279797s
Jan 24 06:43:59.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:01.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 26.320266809s
Jan 24 06:44:01.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:03.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 28.319811253s
Jan 24 06:44:03.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:05.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 30.320342123s
Jan 24 06:44:05.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:07.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 32.319499844s
Jan 24 06:44:07.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:09.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 34.319246116s
Jan 24 06:44:09.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:11.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 36.319660937s
Jan 24 06:44:11.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:13.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 38.320276733s
Jan 24 06:44:13.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:15.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 40.319452538s
Jan 24 06:44:15.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:17.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 42.319191881s
Jan 24 06:44:17.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:19.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 44.319680797s
Jan 24 06:44:19.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:21.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 46.319642338s
Jan 24 06:44:21.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:23.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 48.319822371s
Jan 24 06:44:23.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:25.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 50.319387032s
Jan 24 06:44:25.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:27.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 52.319033924s
Jan 24 06:44:27.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:29.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 54.319675999s
Jan 24 06:44:29.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:31.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 56.319334514s
Jan 24 06:44:31.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:33.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 58.319049884s
Jan 24 06:44:33.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:35.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.319802301s
Jan 24 06:44:35.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:37.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.31954346s
Jan 24 06:44:37.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:39.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.31928129s
Jan 24 06:44:39.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:41.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.319012228s
Jan 24 06:44:41.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:43.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.31936774s
Jan 24 06:44:43.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:45.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.319669227s
Jan 24 06:44:45.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:47.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.31925701s
Jan 24 06:44:47.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:49.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.319997031s
Jan 24 06:44:49.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:51.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.319766789s
Jan 24 06:44:51.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:53.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.319122404s
Jan 24 06:44:53.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:55.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.319816364s
Jan 24 06:44:55.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:57.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.319552488s
Jan 24 06:44:57.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:59.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.319324554s
Jan 24 06:44:59.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:01.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.319658169s
Jan 24 06:45:01.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:03.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.319188108s
Jan 24 06:45:03.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:05.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.320100684s
Jan 24 06:45:05.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:07.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.319712498s
Jan 24 06:45:07.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:09.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.31935972s
Jan 24 06:45:09.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:11.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.319639022s
Jan 24 06:45:11.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:13.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.32010909s
Jan 24 06:45:13.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:15.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.319852169s
Jan 24 06:45:15.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:17.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.319292187s
Jan 24 06:45:17.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:19.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.319784765s
Jan 24 06:45:19.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:21.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.319535121s
Jan 24 06:45:21.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:23.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.319302183s
Jan 24 06:45:23.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:25.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.31955929s
Jan 24 06:45:25.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:27.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.319375818s
Jan 24 06:45:27.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:29.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.31963731s
Jan 24 06:45:29.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:31.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.319414598s
Jan 24 06:45:31.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:33.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.319663109s
Jan 24 06:45:33.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:35.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.319321295s
Jan 24 06:45:35.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:37.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.319349324s
Jan 24 06:45:37.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:39.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.319529413s
Jan 24 06:45:39.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:41.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.319604753s
Jan 24 06:45:41.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:43.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.319787443s
Jan 24 06:45:43.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:45.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.319430584s
Jan 24 06:45:45.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:47.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.31977378s
Jan 24 06:45:47.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:49.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.319622135s
Jan 24 06:45:49.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:51.415: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.320843345s
Jan 24 06:45:51.415: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:53.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.319692849s
Jan 24 06:45:53.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:55.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.319132633s
Jan 24 06:45:55.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:57.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.319444601s
Jan 24 06:45:57.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:59.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.31923501s
Jan 24 06:45:59.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:01.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.319587391s
Jan 24 06:46:01.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:03.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.319594422s
Jan 24 06:46:03.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:05.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.320500853s
Jan 24 06:46:05.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:07.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.320561789s
Jan 24 06:46:07.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:09.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.319443042s
Jan 24 06:46:09.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:11.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.319468534s
Jan 24 06:46:11.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:13.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.319379092s
Jan 24 06:46:13.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:15.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.319298035s
Jan 24 06:46:15.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:17.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.319424804s
Jan 24 06:46:17.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:19.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.319541246s
Jan 24 06:46:19.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:21.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.319807029s
Jan 24 06:46:21.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:23.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.319824505s
Jan 24 06:46:23.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:25.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.320639302s
Jan 24 06:46:25.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:27.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.319685867s
Jan 24 06:46:27.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:29.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.319098418s
Jan 24 06:46:29.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:31.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.319246062s
Jan 24 06:46:31.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:33.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.319683964s
Jan 24 06:46:33.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:35.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.319315709s
Jan 24 06:46:35.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:37.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.32010486s
Jan 24 06:46:37.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:39.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.320088803s
Jan 24 06:46:39.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:41.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.319654971s
Jan 24 06:46:41.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:43.415: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.321265243s
Jan 24 06:46:43.415: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:45.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.319541931s
Jan 24 06:46:45.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:47.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.320274516s
Jan 24 06:46:47.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:49.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.320141589s
Jan 24 06:46:49.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:51.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.320116945s
Jan 24 06:46:51.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:53.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.32005717s
Jan 24 06:46:53.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:55.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.319509285s
Jan 24 06:46:55.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:57.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.319307064s
Jan 24 06:46:57.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:59.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.319544229s
Jan 24 06:46:59.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:01.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.320213251s
Jan 24 06:47:01.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:03.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.319396533s
Jan 24 06:47:03.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:05.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.319809886s
Jan 24 06:47:05.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:07.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.319976273s
Jan 24 06:47:07.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:09.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.319340228s
Jan 24 06:47:09.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:11.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.319620606s
Jan 24 06:47:11.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:13.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.319330571s
Jan 24 06:47:13.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:15.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.320239134s
Jan 24 06:47:15.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:17.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.319781505s
Jan 24 06:47:17.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:19.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.319499259s
Jan 24 06:47:19.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:21.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.319415365s
Jan 24 06:47:21.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:23.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.319475156s
Jan 24 06:47:23.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:25.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.319298499s
Jan 24 06:47:25.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:27.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.319909374s
Jan 24 06:47:27.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:29.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.320141618s
Jan 24 06:47:29.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:31.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.319847131s
Jan 24 06:47:31.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:33.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.319503298s
Jan 24 06:47:33.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:35.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.319004628s
Jan 24 06:47:35.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:37.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.319620434s
Jan 24 06:47:37.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:39.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.3191767s
Jan 24 06:47:39.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:41.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.319588497s
Jan 24 06:47:41.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:43.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.319315782s
Jan 24 06:47:43.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:45.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.319632009s
Jan 24 06:47:45.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:47.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.319600231s
Jan 24 06:47:47.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:49.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.319765955s
Jan 24 06:47:49.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:51.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.319476639s
Jan 24 06:47:51.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:53.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.319271643s
Jan 24 06:47:53.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:55.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.320135797s
Jan 24 06:47:55.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:57.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.319695323s
Jan 24 06:47:57.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:59.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.319379455s
Jan 24 06:47:59.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:01.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.320084647s
Jan 24 06:48:01.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:03.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.319367041s
Jan 24 06:48:03.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:05.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.31957019s
Jan 24 06:48:05.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:07.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.320044102s
Jan 24 06:48:07.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:09.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.319860588s
Jan 24 06:48:09.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:11.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.319686225s
Jan 24 06:48:11.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:13.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.319891873s
Jan 24 06:48:13.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:15.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.319595299s
Jan 24 06:48:15.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:17.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.319333873s
Jan 24 06:48:17.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:19.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.319862271s
Jan 24 06:48:19.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:21.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.319364952s
Jan 24 06:48:21.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:23.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.319639117s
Jan 24 06:48:23.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:25.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.319565456s
Jan 24 06:48:25.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:27.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.319337654s
Jan 24 06:48:27.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:29.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.320126655s
Jan 24 06:48:29.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:31.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.320115272s
Jan 24 06:48:31.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:33.413: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.319309351s
Jan 24 06:48:33.413: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:35.414: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.319827194s
Jan 24 06:48:35.414: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:35.573: INFO: Pod "netserver-3": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.479165485s
Jan 24 06:48:35.573: INFO: The phase of Pod netserver-3 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:35.574: INFO: Unexpected error:
<*pod.timeoutError | 0xc004b1d140>: {
msg: "timed out while waiting for pod nettest-2681/netserver-3 to be running and ready",
observedObjects: [
<*v1.Pod | 0xc0046eed80>{
TypeMeta: {Kind: "", APIVersion: ""},
ObjectMeta: {
Name: "netserver-3",
GenerateName: "",
Namespace: "nettest-2681",
SelfLink: "",
UID: "0499c3ec-1710-48e8-a66e-17189abdfed1",
ResourceVersion: "30236",
Generation: 0,
CreationTimestamp: {
Time: {
wall: 0,
ext: 63810139363,
loc: {
name: "Local",
zone: [
{name: "UTC", offset: 0, isDST: false},
],
tx: [
{
when: -9223372036854775808,
index: 0,
isstd: false,
isutc: false,
},
],
extend: "UTC0",
cacheStart: 9223372036854775807,
cacheEnd: 9223372036854775807,
cacheZone: {name: "UTC", offset: 0, isDST: false},
},
},
},
DeletionTimestamp: nil,
DeletionGracePeriodSeconds: nil,
Labels: {
"selector-02cb5c5d-1d54-4d27-8c16-4c4309f3f630": "true",
},
Annotations: nil,
OwnerReferences: nil,
Finalizers: nil,
ManagedFields: [
{
Manager: "e2e.test",
Operation: "Update",
APIVersion: "v1",
Time: {
Time: {
wall: 0,
ext: 63810139363,
loc: {
name: "Local",
zone: [...],
tx: [...],
extend: "UTC0",
cacheStart: 9223372036854775807,
cacheEnd: 9223372036854775807,
cacheZone: {name: ..., offset: ..., isDST: ...},
},
},
},
FieldsType: "FieldsV1",
FieldsV1: {
Raw: "{\"f:metadata\":{\"f:labels\":{\".\":{},\"f:selector-02cb5c5d-1d54-4d27-8c16-4c4309f3f630\":{}}},\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"webserver\\\"}\":{\".\":{},\"f:args\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:livenessProbe\":{\".\":{},\"f:failureThreshold\":{},\"f:httpGet\":{\".\":{},\"f:path\":{},\"f:port\":{},\"f:scheme\":{}},\"f:initialDelaySeconds\":{},\"f:periodSeconds\":{},\"f:successThreshold\":{},\"f:timeoutSeconds\":{}},\"f:name\":{},\"f:ports\":{\".\":{},\"k:{\\\"containerPort\\\":8081,\\\"protocol\\\":\\\"UDP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:name\":{},\"f:protocol\":{}},\"k:{\\\"containerPort\\\":8083,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:name\":{},\"f:protocol\":{}}},\"f:readinessProbe\":{\".\":{},\"f:failureThreshold\":{},\"f:httpGet\":{\".\":{},\"f:path\":{},\"f:port\":{},\"f:scheme\":{}},\"f:initialDelaySeconds\":{},\"f:periodSeconds\":{},\"f:successThreshold\":{},\"f:timeoutSeconds\":{}},\"f:resources\":{},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:nodeSelector\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:terminationGracePeriodSeconds\":{}}}",
},
Subresource: "",
},
{
Manager: "kubelet",
Operation: "Update",
APIVersion: "v1",
Time: {
Time: {
wall: 0,
ext: 63810139363,
loc: {
name: "Local",
zone: [...],
tx: [...],
extend: "UTC0",
cacheStart: 9223372036854775807,
cacheEnd: 9223372036854775807,
cacheZone: {name: ..., offset: ..., isDST: ...},
},
},
},
FieldsType: "FieldsV1",
FieldsV1: {
Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}",
},
Subresource: "status",
},
],
},
Spec: {
Volumes: [
{
Name: "kube-api-access-5sprs",
VolumeSource: {
HostPath: nil,
EmptyDir: nil,
GCEPersistentDisk: nil,
AWSElasticBlockStore: nil,
GitRepo: nil,
Secret: nil,
NFS: nil,
ISCSI: nil,
Glusterfs: nil,
PersistentVolumeClaim: nil,
RBD: nil,
FlexVolume: nil,
Cinder: nil,
CephFS: nil,
Flocker: nil,
DownwardAPI: nil,
FC: nil,
AzureFile: nil,
ConfigMap: nil,
VsphereVolume: nil,
Quobyte: nil,
AzureDisk: nil,
PhotonPersistentDisk: nil,
Projected: {
Sources: [
{
Secret: ...,
DownwardAPI: ...,
ConfigMap: ...,
ServiceAccountToken: ...,
},
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
Jan 24 06:48:35.574: FAIL: timed out while waiting for pod nettest-2681/netserver-3 to be running and ready
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc0036b2000, {0x75cb852, 0x9}, 0xc005f03560)
test/e2e/framework/network/utils.go:866 +0x1d0
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc0036b2000, 0x7ff5d9a87758?)
test/e2e/framework/network/utils.go:763 +0x55
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc0036b2000, 0x3e?)
test/e2e/framework/network/utils.go:778 +0x3e
k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc0004085a0, {0x0, 0x0, 0x0?})
test/e2e/framework/network/utils.go:131 +0x125
k8s.io/kubernetes/test/e2e/network.glob..func22.6.18()
test/e2e/network/networking.go:452 +0x36
[AfterEach] [sig-network] Networking
test/e2e/framework/node/init/init.go:32
Jan 24 06:48:35.574: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-network] Networking
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-network] Networking
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/24/23 06:48:35.735�[0m
�[1mSTEP:�[0m Collecting events from namespace "nettest-2681". �[38;5;243m01/24/23 06:48:35.735�[0m
�[1mSTEP:�[0m Found 29 events. �[38;5;243m01/24/23 06:48:35.895�[0m
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:42:43 +0000 UTC - event for netserver-0: {default-scheduler } Scheduled: Successfully assigned nettest-2681/netserver-0 to i-036f641c68d5ad392
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:42:43 +0000 UTC - event for netserver-1: {default-scheduler } Scheduled: Successfully assigned nettest-2681/netserver-1 to i-0697a70b9aa95b7c0
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:42:43 +0000 UTC - event for netserver-1: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b9293477800d25ab34019ad78ff7d4d4ba9fbc497906b5391225e41df3ab5668": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:42:43 +0000 UTC - event for netserver-2: {default-scheduler } Scheduled: Successfully assigned nettest-2681/netserver-2 to i-0e0a289a9d5c333a0
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:42:43 +0000 UTC - event for netserver-3: {default-scheduler } Scheduled: Successfully assigned nettest-2681/netserver-3 to i-0f5ee4c33b6799e76
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:42:44 +0000 UTC - event for netserver-0: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "87ea951bab6f4f0cf476b0736a5e1aa528ea05594a7fac066a20b6be8511260b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:42:44 +0000 UTC - event for netserver-2: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ee380803d58be906f7e7864a2d70c05b4e8100b5abdaaea7a1249c6cc2a42ffb": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:42:44 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0478067da1744417b7ea00765622888cd2c07533f1a6ef0fe671afd2a1ce68fa": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:42:55 +0000 UTC - event for netserver-1: {kubelet i-0697a70b9aa95b7c0} Created: Created container webserver
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:42:55 +0000 UTC - event for netserver-1: {kubelet i-0697a70b9aa95b7c0} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:42:55 +0000 UTC - event for netserver-1: {kubelet i-0697a70b9aa95b7c0} Started: Started container webserver
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:42:55 +0000 UTC - event for netserver-2: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "75318e53059bc7c202172c73755781f8c9771165a816360ac92f4bc272551859": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:42:57 +0000 UTC - event for netserver-0: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "aa939d3890aca8bdf1b012588c8402bd0f5ae0a7e5d8099621f9bb870e15aab1": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:42:59 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "110795485d4ed78946cc8eec82e5b07d52276cad71e8dfe2a5c5ef9397f8b444": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:43:07 +0000 UTC - event for netserver-2: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "66d25929e8164cddd6ee0a93e3a7a7fe544e2ce6526a3081fc8c600adf936c6f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:43:10 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "83b3664ad4e197581b51b10ecd0fe678d1d50ac5b6700b2222766503df0f13b9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:43:13 +0000 UTC - event for netserver-0: {kubelet i-036f641c68d5ad392} Started: Started container webserver
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:43:13 +0000 UTC - event for netserver-0: {kubelet i-036f641c68d5ad392} Created: Created container webserver
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:43:13 +0000 UTC - event for netserver-0: {kubelet i-036f641c68d5ad392} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:43:19 +0000 UTC - event for netserver-2: {kubelet i-0e0a289a9d5c333a0} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:43:19 +0000 UTC - event for netserver-2: {kubelet i-0e0a289a9d5c333a0} Created: Created container webserver
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:43:20 +0000 UTC - event for netserver-2: {kubelet i-0e0a289a9d5c333a0} Started: Started container webserver
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:43:24 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8d709beb331f7c744996f110b88cc95a419ea979c42a4a7c33a7981cbceda4b8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:43:37 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ff43c153f174c51858a1fb7edcf4f876075ea257ffdb356ece0aacb4b714c479": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:43:48 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "663d25ed9b25681e8f769aabcbf923bbdf69c9e18e01f8671ff7e4630ac2bfd5": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:44:00 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "53f20707d579c62b07bfef7be09bcd73a5360710fa7a08d2f243fcbf0113d39b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:44:13 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "029375682169873f3b094dbbcee938b13e7304f205a40cf95ba0931c611f2b5e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:44:28 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "78eedeb64d78e5293a39b46da2708c3d3badc887464023a400b2d373bb9034fa": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:35.895: INFO: At 2023-01-24 06:44:40 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c425f928e170a443a38f250191f0964094c9298717b44e0625cc52a01683be88": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:48:36.054: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 24 06:48:36.054: INFO: netserver-0 i-036f641c68d5ad392 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:42:43 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:24 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:24 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:42:43 +0000 UTC }]
Jan 24 06:48:36.054: INFO: netserver-1 i-0697a70b9aa95b7c0 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:42:43 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:13 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:13 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:42:43 +0000 UTC }]
Jan 24 06:48:36.054: INFO: netserver-2 i-0e0a289a9d5c333a0 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:42:43 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:34 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:34 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:42:43 +0000 UTC }]
Jan 24 06:48:36.055: INFO: netserver-3 i-0f5ee4c33b6799e76 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:42:43 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:42:43 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:42:43 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:42:43 +0000 UTC }]
Jan 24 06:48:36.055: INFO:
Jan 24 06:48:36.717: INFO: Unable to fetch nettest-2681/netserver-3/webserver logs: the server rejected our request for an unknown reason (get pods netserver-3)
Jan 24 06:48:36.877: INFO:
Logging node info for node i-036f641c68d5ad392
Jan 24 06:48:37.037: INFO: Node Info: &Node{ObjectMeta:{i-036f641c68d5ad392 50721840-3696-4207-81a9-8cd089fbb539 31690 0 2023-01-24 06:27:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-036f641c68d5ad392 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-036f641c68d5ad392 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.254 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-8398":"i-036f641c68d5ad392","ebs.csi.aws.com":"i-036f641c68d5ad392"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:44:10 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-24 06:44:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-036f641c68d5ad392,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:18 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:18 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:18 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:44:18 +0000 UTC,LastTransitionTime:2023-01-24 06:27:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.254,},NodeAddress{Type:ExternalIP,Address:15.165.237.223,},NodeAddress{Type:InternalDNS,Address:i-036f641c68d5ad392.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-036f641c68d5ad392.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-165-237-223.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2732c784a9bb7e64a787956c82cf85,SystemUUID:ec2732c7-84a9-bb7e-64a7-87956c82cf85,BootID:bee5ed40-42b1-475e-b72b-16d715de535a,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-8398^8152cf6c-9bb2-11ed-ba50-629dc9cbc81c kubernetes.io/csi/ebs.csi.aws.com^vol-0269089ae9a4173b7],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0269089ae9a4173b7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-8398^8152cf6c-9bb2-11ed-ba50-629dc9cbc81c,DevicePath:,},},Config:nil,},}
Jan 24 06:48:37.037: INFO:
Logging kubelet events for node i-036f641c68d5ad392
Jan 24 06:48:37.199: INFO:
Logging pods the kubelet thinks is on node i-036f641c68d5ad392
Jan 24 06:48:37.368: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6twph started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:37.368: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-m4sr7 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:37.368: INFO: csi-mockplugin-0 started at 2023-01-24 06:47:12 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container csi-provisioner ready: false, restart count 0
Jan 24 06:48:37.368: INFO: Container driver-registrar ready: false, restart count 0
Jan 24 06:48:37.368: INFO: Container mock ready: false, restart count 0
Jan 24 06:48:37.368: INFO: hostexec-i-036f641c68d5ad392-g49z8 started at 2023-01-24 06:47:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container agnhost-container ready: true, restart count 0
Jan 24 06:48:37.368: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2mkph started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:37.368: INFO: rs-6s4df started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container donothing ready: false, restart count 0
Jan 24 06:48:37.368: INFO: csi-hostpathplugin-0 started at 2023-01-24 06:44:58 +0000 UTC (0+7 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container csi-attacher ready: false, restart count 0
Jan 24 06:48:37.368: INFO: Container csi-provisioner ready: false, restart count 0
Jan 24 06:48:37.368: INFO: Container csi-resizer ready: false, restart count 0
Jan 24 06:48:37.368: INFO: Container csi-snapshotter ready: false, restart count 0
Jan 24 06:48:37.368: INFO: Container hostpath ready: false, restart count 0
Jan 24 06:48:37.368: INFO: Container liveness-probe ready: false, restart count 0
Jan 24 06:48:37.368: INFO: Container node-driver-registrar ready: false, restart count 0
Jan 24 06:48:37.368: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-8sfzb started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:48:37.368: INFO: rs-zmrj2 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container donothing ready: false, restart count 0
Jan 24 06:48:37.368: INFO: csi-hostpathplugin-0 started at 2023-01-24 06:42:31 +0000 UTC (0+7 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container csi-attacher ready: true, restart count 0
Jan 24 06:48:37.368: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:48:37.368: INFO: Container csi-resizer ready: true, restart count 0
Jan 24 06:48:37.368: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 24 06:48:37.368: INFO: Container hostpath ready: true, restart count 0
Jan 24 06:48:37.368: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:48:37.368: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:48:37.368: INFO: netserver-0 started at 2023-01-24 06:42:43 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container webserver ready: true, restart count 0
Jan 24 06:48:37.368: INFO: ebs-csi-node-6z65r started at 2023-01-24 06:27:40 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:48:37.368: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:48:37.368: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:48:37.368: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cpp7h started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:37.368: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6hnbl started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:37.368: INFO: pod-subpath-test-dynamicpv-s6jk started at 2023-01-24 06:44:10 +0000 UTC (1+2 container statuses recorded)
Jan 24 06:48:37.368: INFO: Init container test-init-subpath-dynamicpv-s6jk ready: false, restart count 0
Jan 24 06:48:37.368: INFO: Container test-container-subpath-dynamicpv-s6jk ready: false, restart count 0
Jan 24 06:48:37.368: INFO: Container test-container-volume-dynamicpv-s6jk ready: false, restart count 0
Jan 24 06:48:37.368: INFO: service-headless-btk74 started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container service-headless ready: false, restart count 0
Jan 24 06:48:37.368: INFO: pod-da9584f9-0d79-4167-ad37-ca206c3ac911 started at 2023-01-24 06:43:12 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container write-pod ready: false, restart count 0
Jan 24 06:48:37.368: INFO: csi-mockplugin-resizer-0 started at 2023-01-24 06:47:12 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container csi-resizer ready: false, restart count 0
Jan 24 06:48:37.368: INFO: pod-subpath-test-preprovisionedpv-l2p7 started at 2023-01-24 06:48:02 +0000 UTC (1+2 container statuses recorded)
Jan 24 06:48:37.368: INFO: Init container test-init-subpath-preprovisionedpv-l2p7 ready: false, restart count 0
Jan 24 06:48:37.368: INFO: Container test-container-subpath-preprovisionedpv-l2p7 ready: false, restart count 0
Jan 24 06:48:37.368: INFO: Container test-container-volume-preprovisionedpv-l2p7 ready: false, restart count 0
Jan 24 06:48:37.368: INFO: pod-handle-http-request started at 2023-01-24 06:48:15 +0000 UTC (0+2 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container container-handle-http-request ready: false, restart count 0
Jan 24 06:48:37.368: INFO: Container container-handle-https-request ready: false, restart count 0
Jan 24 06:48:37.368: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-h5mpm started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:37.368: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-tcp26 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:37.368: INFO: cilium-9ht2g started at 2023-01-24 06:27:40 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:48:37.368: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:48:37.368: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-t9b6f started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:37.368: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-lc7s4 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:37.368: INFO: rand-non-local-gxd9p started at 2023-01-24 06:42:31 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container c ready: false, restart count 0
Jan 24 06:48:37.368: INFO: netserver-0 started at 2023-01-24 06:43:38 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container webserver ready: true, restart count 0
Jan 24 06:48:37.368: INFO: rs-76gck started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:37.368: INFO: Container donothing ready: false, restart count 0
Jan 24 06:48:37.965: INFO:
Latency metrics for node i-036f641c68d5ad392
Jan 24 06:48:37.965: INFO:
Logging node info for node i-0697a70b9aa95b7c0
Jan 24 06:48:38.125: INFO: Node Info: &Node{ObjectMeta:{i-0697a70b9aa95b7c0 d0a3c976-eb07-4323-b48b-ec4945de7410 31075 0 2023-01-24 06:27:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0697a70b9aa95b7c0 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0697a70b9aa95b7c0 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.153 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0697a70b9aa95b7c0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:43:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0697a70b9aa95b7c0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:43:35 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:43:35 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:43:35 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:43:35 +0000 UTC,LastTransitionTime:2023-01-24 06:27:49 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.153,},NodeAddress{Type:ExternalIP,Address:3.38.96.49,},NodeAddress{Type:InternalDNS,Address:i-0697a70b9aa95b7c0.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0697a70b9aa95b7c0.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-38-96-49.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f1e14d470f251c9d042c57478cb8b,SystemUUID:ec2f1e14-d470-f251-c9d0-42c57478cb8b,BootID:578a4c1d-084e-4aeb-92fe-f64671842d82,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:48:38.125: INFO:
Logging kubelet events for node i-0697a70b9aa95b7c0
Jan 24 06:48:38.287: INFO:
Logging pods the kubelet thinks is on node i-0697a70b9aa95b7c0
Jan 24 06:48:38.455: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cdkdr started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:38.455: INFO: rs-w9wjl started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container donothing ready: false, restart count 0
Jan 24 06:48:38.455: INFO: csi-mockplugin-0 started at 2023-01-24 06:44:33 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container csi-provisioner ready: false, restart count 0
Jan 24 06:48:38.455: INFO: Container driver-registrar ready: false, restart count 0
Jan 24 06:48:38.455: INFO: Container mock ready: false, restart count 0
Jan 24 06:48:38.455: INFO: csi-mockplugin-resizer-0 started at 2023-01-24 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container csi-resizer ready: false, restart count 0
Jan 24 06:48:38.455: INFO: coredns-autoscaler-7cb5c5b969-vktlf started at 2023-01-24 06:27:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container autoscaler ready: true, restart count 0
Jan 24 06:48:38.455: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-xbpd8 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:48:38.455: INFO: ebs-csi-node-pkrr4 started at 2023-01-24 06:27:36 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:48:38.455: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:48:38.455: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:48:38.455: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-fgc56 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:38.455: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2v7c5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:38.455: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-4sk4h started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:38.455: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-68w92 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:38.455: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qphds started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:38.455: INFO: webserver-pod started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container agnhost ready: false, restart count 0
Jan 24 06:48:38.455: INFO: hostexec-i-0697a70b9aa95b7c0-g8mhd started at 2023-01-24 06:43:35 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container agnhost-container ready: true, restart count 0
Jan 24 06:48:38.455: INFO: service-headless-xh4pl started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container service-headless ready: false, restart count 0
Jan 24 06:48:38.455: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qtzq5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:38.455: INFO: netserver-1 started at 2023-01-24 06:43:39 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container webserver ready: false, restart count 0
Jan 24 06:48:38.455: INFO: rs-b9vhw started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container donothing ready: false, restart count 0
Jan 24 06:48:38.455: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2tkvs started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:48:38.455: INFO: pod-subpath-test-inlinevolume-qdbt started at 2023-01-24 06:44:17 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Init container init-volume-inlinevolume-qdbt ready: false, restart count 0
Jan 24 06:48:38.455: INFO: Container test-container-subpath-inlinevolume-qdbt ready: false, restart count 0
Jan 24 06:48:38.455: INFO: netserver-1 started at 2023-01-24 06:42:43 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container webserver ready: true, restart count 0
Jan 24 06:48:38.455: INFO: csi-mockplugin-attacher-0 started at 2023-01-24 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container csi-attacher ready: false, restart count 0
Jan 24 06:48:38.455: INFO: cilium-dh7hm started at 2023-01-24 06:27:36 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:48:38.455: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:48:38.455: INFO: coredns-559769c974-6wblg started at 2023-01-24 06:27:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container coredns ready: true, restart count 0
Jan 24 06:48:38.455: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-5pcw5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:38.455: INFO: exec-volume-test-preprovisionedpv-zb7d started at 2023-01-24 06:43:48 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:38.455: INFO: Container exec-container-preprovisionedpv-zb7d ready: false, restart count 0
Jan 24 06:48:39.061: INFO:
Latency metrics for node i-0697a70b9aa95b7c0
Jan 24 06:48:39.061: INFO:
Logging node info for node i-0b0b3ac061164cea4
Jan 24 06:48:39.220: INFO: Node Info: &Node{ObjectMeta:{i-0b0b3ac061164cea4 ac00aaf7-4427-49f6-b849-9c812ae80069 33008 0 2023-01-24 06:25:23 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b0b3ac061164cea4 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.51.48 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b0b3ac061164cea4"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-24 06:25:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-24 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:26:22 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:26:22 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-24 06:26:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-24 06:48:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0b0b3ac061164cea4,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892297728 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787440128 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:26:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.48,},NodeAddress{Type:ExternalIP,Address:13.125.222.159,},NodeAddress{Type:InternalDNS,Address:i-0b0b3ac061164cea4.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b0b3ac061164cea4.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-125-222-159.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255807a7dec89d2cc9cfee6d481c20,SystemUUID:ec255807-a7de-c89d-2cc9-cfee6d481c20,BootID:4153aea0-5d07-4f37-a966-c56ef660acfc,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.1],SizeBytes:135178704,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.1],SizeBytes:124995897,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:63920476,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.1],SizeBytes:57661752,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:44339030,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42841305,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4965794,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:48:39.221: INFO:
Logging kubelet events for node i-0b0b3ac061164cea4
Jan 24 06:48:39.383: INFO:
Logging pods the kubelet thinks is on node i-0b0b3ac061164cea4
Jan 24 06:48:39.551: INFO: ebs-csi-node-q4jsj started at 2023-01-24 06:26:07 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:48:39.551: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:48:39.551: INFO: kops-controller-vfj77 started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:39.551: INFO: Container kops-controller ready: true, restart count 0
Jan 24 06:48:39.551: INFO: dns-controller-69987775c6-vzqc8 started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:39.551: INFO: Container dns-controller ready: true, restart count 0
Jan 24 06:48:39.551: INFO: ebs-csi-controller-5bd98b456f-kswjq started at 2023-01-24 06:26:07 +0000 UTC (0+5 container statuses recorded)
Jan 24 06:48:39.551: INFO: Container csi-attacher ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Container csi-resizer ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:48:39.551: INFO: kube-apiserver-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+2 container statuses recorded)
Jan 24 06:48:39.551: INFO: Container healthcheck ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Container kube-apiserver ready: true, restart count 3
Jan 24 06:48:39.551: INFO: aws-cloud-controller-manager-4n9pv started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:39.551: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 24 06:48:39.551: INFO: kube-controller-manager-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:39.551: INFO: Container kube-controller-manager ready: true, restart count 4
Jan 24 06:48:39.551: INFO: kube-scheduler-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:39.551: INFO: Container kube-scheduler ready: true, restart count 0
Jan 24 06:48:39.551: INFO: cilium-w225f started at 2023-01-24 06:26:07 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:48:39.551: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:48:39.551: INFO: cilium-operator-5dd44dc49f-rn9bl started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:39.551: INFO: Container cilium-operator ready: true, restart count 0
Jan 24 06:48:39.551: INFO: etcd-manager-events-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (10+1 container statuses recorded)
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Container etcd-manager ready: true, restart count 0
Jan 24 06:48:39.551: INFO: etcd-manager-main-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (10+1 container statuses recorded)
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 24 06:48:39.551: INFO: Container etcd-manager ready: true, restart count 0
Jan 24 06:48:40.087: INFO:
Latency metrics for node i-0b0b3ac061164cea4
Jan 24 06:48:40.087: INFO:
Logging node info for node i-0e0a289a9d5c333a0
Jan 24 06:48:40.247: INFO: Node Info: &Node{ObjectMeta:{i-0e0a289a9d5c333a0 9194794b-d78a-4a30-bbec-b76df63f68f1 31709 0 2023-01-24 06:27:40 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0e0a289a9d5c333a0 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0e0a289a9d5c333a0 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.62 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0e0a289a9d5c333a0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:44:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0e0a289a9d5c333a0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:10 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:10 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:10 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:44:10 +0000 UTC,LastTransitionTime:2023-01-24 06:27:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.62,},NodeAddress{Type:ExternalIP,Address:54.180.158.26,},NodeAddress{Type:InternalDNS,Address:i-0e0a289a9d5c333a0.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0e0a289a9d5c333a0.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-180-158-26.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2281a7b00c77d27e8055a22bf9facf,SystemUUID:ec2281a7-b00c-77d2-7e80-55a22bf9facf,BootID:be2e68ba-6839-4b53-847f-22f29f0a7360,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:48:40.248: INFO:
Logging kubelet events for node i-0e0a289a9d5c333a0
Jan 24 06:48:40.410: INFO:
Logging pods the kubelet thinks is on node i-0e0a289a9d5c333a0
Jan 24 06:48:40.577: INFO: alpine-nnp-false-2c8bfdf9-3bfb-42e5-835c-fb8faa43651e started at 2023-01-24 06:46:40 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container alpine-nnp-false-2c8bfdf9-3bfb-42e5-835c-fb8faa43651e ready: false, restart count 0
Jan 24 06:48:40.577: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-9hr5l started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:40.577: INFO: netserver-2 started at 2023-01-24 06:43:39 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container webserver ready: true, restart count 0
Jan 24 06:48:40.577: INFO: test-orphan-deployment-7f5969cbc7-b98k5 started at 2023-01-24 06:44:00 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container httpd ready: false, restart count 0
Jan 24 06:48:40.577: INFO: service-headless-h4z88 started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container service-headless ready: false, restart count 0
Jan 24 06:48:40.577: INFO: cilium-cvt86 started at 2023-01-24 06:27:41 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:48:40.577: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:48:40.577: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-224qm started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:40.577: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-z9lc4 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:40.577: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-7nhf8 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:40.577: INFO: rs-l5fgd started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container donothing ready: false, restart count 0
Jan 24 06:48:40.577: INFO: netserver-2 started at 2023-01-24 06:42:43 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container webserver ready: true, restart count 0
Jan 24 06:48:40.577: INFO: pod-d2fe4cd9-5425-48c9-8a72-8417570ba681 started at 2023-01-24 06:47:54 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container test-container ready: false, restart count 0
Jan 24 06:48:40.577: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-867tg started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:40.577: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-zv4q6 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:40.577: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-tbbms started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:40.577: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-w7m2k started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:48:40.577: INFO: ebs-csi-node-v6prc started at 2023-01-24 06:27:41 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:48:40.577: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:48:40.577: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:48:40.577: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-ts8xr started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:40.577: INFO: rs-rsqdt started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container donothing ready: true, restart count 0
Jan 24 06:48:40.577: INFO: rc-test-qrc82 started at 2023-01-24 06:48:11 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container rc-test ready: false, restart count 0
Jan 24 06:48:40.577: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-bkjzk started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:40.577: INFO: csi-mockplugin-0 started at 2023-01-24 06:48:18 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container csi-provisioner ready: false, restart count 0
Jan 24 06:48:40.577: INFO: Container driver-registrar ready: false, restart count 0
Jan 24 06:48:40.577: INFO: Container mock ready: false, restart count 0
Jan 24 06:48:40.577: INFO: rs-znpr8 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container donothing ready: false, restart count 0
Jan 24 06:48:40.577: INFO: csi-mockplugin-attacher-0 started at 2023-01-24 06:48:19 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:40.577: INFO: Container csi-attacher ready: false, restart count 0
Jan 24 06:48:41.284: INFO:
Latency metrics for node i-0e0a289a9d5c333a0
Jan 24 06:48:41.284: INFO:
Logging node info for node i-0f5ee4c33b6799e76
Jan 24 06:48:41.444: INFO: Node Info: &Node{ObjectMeta:{i-0f5ee4c33b6799e76 34c5957d-82ce-41f2-824d-90a78fa8bbba 31698 0 2023-01-24 06:27:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0f5ee4c33b6799e76 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0f5ee4c33b6799e76 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.57.132 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7943":"i-0f5ee4c33b6799e76","ebs.csi.aws.com":"i-0f5ee4c33b6799e76"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:44:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0f5ee4c33b6799e76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:19 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:19 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:19 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:44:19 +0000 UTC,LastTransitionTime:2023-01-24 06:27:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.57.132,},NodeAddress{Type:ExternalIP,Address:3.36.124.131,},NodeAddress{Type:InternalDNS,Address:i-0f5ee4c33b6799e76.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f5ee4c33b6799e76.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-36-124-131.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bcf8e071d2b94271b14b41c978c,SystemUUID:ec2f5bcf-8e07-1d2b-9427-1b14b41c978c,BootID:400ceb00-7274-4349-89f8-a3cb13aed9c2,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:48:41.444: INFO:
Logging kubelet events for node i-0f5ee4c33b6799e76
Jan 24 06:48:41.611: INFO:
Logging pods the kubelet thinks is on node i-0f5ee4c33b6799e76
Jan 24 06:48:41.779: INFO: hostexec-i-0f5ee4c33b6799e76-g6qhd started at 2023-01-24 06:43:32 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container agnhost-container ready: true, restart count 0
Jan 24 06:48:41.779: INFO: hostexec-i-0f5ee4c33b6799e76-26jmv started at 2023-01-24 06:43:55 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container agnhost-container ready: true, restart count 0
Jan 24 06:48:41.779: INFO: pod-e22bfd9d-fd2c-4c39-bf88-7771a5c4ec25 started at 2023-01-24 06:44:00 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container write-pod ready: false, restart count 0
Jan 24 06:48:41.779: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qmz8q started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:41.779: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-xl6bg started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:41.779: INFO: rs-g6rhr started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container donothing ready: false, restart count 0
Jan 24 06:48:41.779: INFO: rs-8b8n9 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container donothing ready: false, restart count 0
Jan 24 06:48:41.779: INFO: pvc-volume-tester-slzmq started at 2023-01-24 06:44:20 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container volume-tester ready: false, restart count 0
Jan 24 06:48:41.779: INFO: cilium-ql489 started at 2023-01-24 06:27:40 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:48:41.779: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:48:41.779: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-hwktw started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:41.779: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cfnfh started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:41.779: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-7lw99 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:41.779: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-w9729 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:41.779: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-lvvw2 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:41.779: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-wjt45 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:41.779: INFO: coredns-559769c974-5gltc started at 2023-01-24 06:28:16 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container coredns ready: true, restart count 0
Jan 24 06:48:41.779: INFO: pod-ca8a4ceb-b094-43f3-aee3-215db13281b7 started at 2023-01-24 06:43:37 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container write-pod ready: false, restart count 0
Jan 24 06:48:41.779: INFO: csi-mockplugin-0 started at 2023-01-24 06:43:09 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:48:41.779: INFO: Container driver-registrar ready: true, restart count 0
Jan 24 06:48:41.779: INFO: Container mock ready: true, restart count 0
Jan 24 06:48:41.779: INFO: netserver-3 started at 2023-01-24 06:43:39 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container webserver ready: false, restart count 0
Jan 24 06:48:41.779: INFO: ebs-csi-node-vjhdb started at 2023-01-24 06:27:40 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:48:41.779: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:48:41.779: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:48:41.779: INFO: netserver-3 started at 2023-01-24 06:42:43 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container webserver ready: false, restart count 0
Jan 24 06:48:41.779: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-r5vgx started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:41.779: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6lkqq started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:48:41.779: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:48:42.362: INFO:
Latency metrics for node i-0f5ee4c33b6799e76
[DeferCleanup (Each)] [sig-network] Networking
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "nettest-2681" for this suite. �[38;5;243m01/24/23 06:48:42.362�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sNetworking\sGranular\sChecks\:\sServices\sshould\sfunction\sfor\sendpoint\-Service\:\sudp$'
test/e2e/framework/network/utils.go:866
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc0039cee00, {0x75cb852, 0x9}, 0xc00393d650)
test/e2e/framework/network/utils.go:866 +0x1d0
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc0039cee00, 0x7fe0d847fc10?)
test/e2e/framework/network/utils.go:763 +0x55
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc0039cee00, 0x3e?)
test/e2e/framework/network/utils.go:778 +0x3e
k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc000e7d590, {0x0, 0x0, 0x7fe101cb5f18?})
test/e2e/framework/network/utils.go:131 +0x125
k8s.io/kubernetes/test/e2e/network.glob..func22.6.8()
test/e2e/network/networking.go:251 +0x36
[BeforeEach] [sig-network] Networking
set up framework | framework.go:178
�[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/24/23 06:43:37.396�[0m
Jan 24 06:43:37.396: INFO: >>> kubeConfig: /root/.kube/config
�[1mSTEP:�[0m Building a namespace api object, basename nettest �[38;5;243m01/24/23 06:43:37.397�[0m
�[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/24/23 06:43:37.875�[0m
�[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/24/23 06:43:38.191�[0m
[BeforeEach] [sig-network] Networking
test/e2e/framework/metrics/init/init.go:31
[It] should function for endpoint-Service: udp
test/e2e/network/networking.go:250
�[1mSTEP:�[0m Performing setup for networking test in namespace nettest-3667 �[38;5;243m01/24/23 06:43:38.507�[0m
�[1mSTEP:�[0m creating a selector �[38;5;243m01/24/23 06:43:38.507�[0m
�[1mSTEP:�[0m Creating the service pods in kubernetes �[38;5;243m01/24/23 06:43:38.507�[0m
Jan 24 06:43:38.507: INFO: Waiting up to 10m0s for all (but 0) nodes to be schedulable
Jan 24 06:43:39.481: INFO: Waiting up to 5m0s for pod "netserver-0" in namespace "nettest-3667" to be "running and ready"
Jan 24 06:43:39.639: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 158.652391ms
Jan 24 06:43:39.639: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:43:41.799: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 2.318472336s
Jan 24 06:43:41.799: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 24 06:43:43.799: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 4.318062194s
Jan 24 06:43:43.799: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 24 06:43:45.798: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 6.317566608s
Jan 24 06:43:45.798: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 24 06:43:47.799: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 8.31802558s
Jan 24 06:43:47.799: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 24 06:43:49.799: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 10.318344661s
Jan 24 06:43:49.799: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 24 06:43:51.799: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 12.318010468s
Jan 24 06:43:51.799: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 24 06:43:53.799: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 14.318248264s
Jan 24 06:43:53.799: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 24 06:43:55.799: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 16.31804266s
Jan 24 06:43:55.799: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 24 06:43:57.798: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 18.317708593s
Jan 24 06:43:57.798: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Jan 24 06:43:59.798: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=true. Elapsed: 20.317553866s
Jan 24 06:43:59.798: INFO: The phase of Pod netserver-0 is Running (Ready = true)
Jan 24 06:43:59.798: INFO: Pod "netserver-0" satisfied condition "running and ready"
Jan 24 06:43:59.957: INFO: Waiting up to 5m0s for pod "netserver-1" in namespace "nettest-3667" to be "running and ready"
Jan 24 06:44:00.115: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 158.42467ms
Jan 24 06:44:00.115: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:02.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2.317861508s
Jan 24 06:44:02.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:04.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4.317385481s
Jan 24 06:44:04.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:06.282: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 6.32568699s
Jan 24 06:44:06.282: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:08.282: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 8.32509432s
Jan 24 06:44:08.282: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:10.277: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 10.32056194s
Jan 24 06:44:10.277: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:12.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 12.318322325s
Jan 24 06:44:12.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:14.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 14.318023917s
Jan 24 06:44:14.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:16.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 16.318061362s
Jan 24 06:44:16.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:18.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 18.318450588s
Jan 24 06:44:18.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:20.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 20.318019214s
Jan 24 06:44:20.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:22.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 22.317889579s
Jan 24 06:44:22.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:24.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 24.317804346s
Jan 24 06:44:24.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:26.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 26.317277249s
Jan 24 06:44:26.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:28.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 28.317265908s
Jan 24 06:44:28.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:30.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 30.317006647s
Jan 24 06:44:30.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:32.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 32.316923587s
Jan 24 06:44:32.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:34.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 34.317143878s
Jan 24 06:44:34.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:36.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 36.318267257s
Jan 24 06:44:36.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:38.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 38.317472021s
Jan 24 06:44:38.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:40.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 40.317118537s
Jan 24 06:44:40.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:42.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 42.317882318s
Jan 24 06:44:42.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:44.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 44.318134592s
Jan 24 06:44:44.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:46.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 46.318116028s
Jan 24 06:44:46.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:48.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 48.318042792s
Jan 24 06:44:48.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:50.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 50.317000274s
Jan 24 06:44:50.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:52.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 52.317775703s
Jan 24 06:44:52.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:54.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 54.317909619s
Jan 24 06:44:54.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:56.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 56.318090872s
Jan 24 06:44:56.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:44:58.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 58.317213616s
Jan 24 06:44:58.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:00.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.318096714s
Jan 24 06:45:00.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:02.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.31805422s
Jan 24 06:45:02.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:04.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.318235507s
Jan 24 06:45:04.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:06.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.317888793s
Jan 24 06:45:06.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:08.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.317046569s
Jan 24 06:45:08.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:10.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.31795661s
Jan 24 06:45:10.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:12.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.317817012s
Jan 24 06:45:12.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:14.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.31781693s
Jan 24 06:45:14.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:16.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.317592569s
Jan 24 06:45:16.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:18.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.3178213s
Jan 24 06:45:18.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:20.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.317717059s
Jan 24 06:45:20.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:22.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.317561471s
Jan 24 06:45:22.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:24.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.317748644s
Jan 24 06:45:24.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:26.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.317547174s
Jan 24 06:45:26.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:28.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.317141686s
Jan 24 06:45:28.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:30.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.317778572s
Jan 24 06:45:30.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:32.279: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.322573573s
Jan 24 06:45:32.279: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:34.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.317921545s
Jan 24 06:45:34.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:36.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.317897255s
Jan 24 06:45:36.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:38.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.316920313s
Jan 24 06:45:38.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:40.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.318326848s
Jan 24 06:45:40.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:42.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.318192047s
Jan 24 06:45:42.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:44.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.317739802s
Jan 24 06:45:44.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:46.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.317499608s
Jan 24 06:45:46.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:48.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.317464432s
Jan 24 06:45:48.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:50.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.318175269s
Jan 24 06:45:50.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:52.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.317919002s
Jan 24 06:45:52.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:54.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.317965268s
Jan 24 06:45:54.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:56.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.317806317s
Jan 24 06:45:56.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:45:58.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.317861408s
Jan 24 06:45:58.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:00.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.317973893s
Jan 24 06:46:00.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:02.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.317672792s
Jan 24 06:46:02.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:04.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.318325825s
Jan 24 06:46:04.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:06.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.317981082s
Jan 24 06:46:06.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:08.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.317383209s
Jan 24 06:46:08.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:10.277: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.320097569s
Jan 24 06:46:10.277: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:12.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.318010779s
Jan 24 06:46:12.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:14.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.317973868s
Jan 24 06:46:14.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:16.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.317784025s
Jan 24 06:46:16.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:18.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.317299312s
Jan 24 06:46:18.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:20.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.317909481s
Jan 24 06:46:20.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:22.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.317822145s
Jan 24 06:46:22.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:24.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.317179918s
Jan 24 06:46:24.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:26.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.318079794s
Jan 24 06:46:26.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:28.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.31771533s
Jan 24 06:46:28.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:30.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.317649722s
Jan 24 06:46:30.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:32.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.317525451s
Jan 24 06:46:32.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:34.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.318149711s
Jan 24 06:46:34.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:36.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.317694923s
Jan 24 06:46:36.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:38.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.318003649s
Jan 24 06:46:38.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:40.276: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.319326536s
Jan 24 06:46:40.276: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:42.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.318095004s
Jan 24 06:46:42.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:44.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.318570553s
Jan 24 06:46:44.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:46.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.317605047s
Jan 24 06:46:46.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:48.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.316998914s
Jan 24 06:46:48.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:50.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.317828393s
Jan 24 06:46:50.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:52.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.317666108s
Jan 24 06:46:52.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:54.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.31785941s
Jan 24 06:46:54.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:56.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.31793454s
Jan 24 06:46:56.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:46:58.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.316851916s
Jan 24 06:46:58.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:00.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.317769174s
Jan 24 06:47:00.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:02.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.317543207s
Jan 24 06:47:02.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:04.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.318011098s
Jan 24 06:47:04.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:06.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.317716851s
Jan 24 06:47:06.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:08.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.317780654s
Jan 24 06:47:08.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:10.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.318245204s
Jan 24 06:47:10.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:12.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.317767373s
Jan 24 06:47:12.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:14.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.318087988s
Jan 24 06:47:14.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:16.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.317892835s
Jan 24 06:47:16.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:18.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.317031928s
Jan 24 06:47:18.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:20.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.317920467s
Jan 24 06:47:20.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:22.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.317845697s
Jan 24 06:47:22.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:24.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.317757942s
Jan 24 06:47:24.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:26.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.317986038s
Jan 24 06:47:26.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:28.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.317118629s
Jan 24 06:47:28.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:30.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.317959247s
Jan 24 06:47:30.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:32.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.317205339s
Jan 24 06:47:32.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:34.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.31766824s
Jan 24 06:47:34.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:36.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.317625547s
Jan 24 06:47:36.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:38.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.31717192s
Jan 24 06:47:38.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:40.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.318038336s
Jan 24 06:47:40.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:42.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.317829722s
Jan 24 06:47:42.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:44.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.317378046s
Jan 24 06:47:44.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:46.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.31804847s
Jan 24 06:47:46.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:48.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.317408608s
Jan 24 06:47:48.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:50.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.317998618s
Jan 24 06:47:50.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:52.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.317595554s
Jan 24 06:47:52.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:54.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.31740642s
Jan 24 06:47:54.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:56.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.317796478s
Jan 24 06:47:56.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:47:58.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.318432159s
Jan 24 06:47:58.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:00.276: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.318808389s
Jan 24 06:48:00.276: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:02.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.317588679s
Jan 24 06:48:02.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:04.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.318063554s
Jan 24 06:48:04.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:06.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.318224062s
Jan 24 06:48:06.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:08.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.317231604s
Jan 24 06:48:08.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:10.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.317477617s
Jan 24 06:48:10.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:12.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.317949329s
Jan 24 06:48:12.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:14.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.317563481s
Jan 24 06:48:14.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:16.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.317894679s
Jan 24 06:48:16.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:18.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.317349138s
Jan 24 06:48:18.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:20.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.317946478s
Jan 24 06:48:20.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:22.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.317633593s
Jan 24 06:48:22.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:24.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.317820592s
Jan 24 06:48:24.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:26.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.317758527s
Jan 24 06:48:26.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:28.278: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.321673874s
Jan 24 06:48:28.278: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:30.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.318506833s
Jan 24 06:48:30.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:32.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.318028147s
Jan 24 06:48:32.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:34.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.317978473s
Jan 24 06:48:34.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:36.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.318358048s
Jan 24 06:48:36.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:38.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.31830388s
Jan 24 06:48:38.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:40.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.317896776s
Jan 24 06:48:40.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:42.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.318073194s
Jan 24 06:48:42.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:44.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.318470576s
Jan 24 06:48:44.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:46.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.318052851s
Jan 24 06:48:46.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:48.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.318331364s
Jan 24 06:48:48.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:50.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.317291183s
Jan 24 06:48:50.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:52.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.317974348s
Jan 24 06:48:52.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:54.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.318452638s
Jan 24 06:48:54.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:56.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.317123331s
Jan 24 06:48:56.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:48:58.274: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.316866327s
Jan 24 06:48:58.274: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:00.275: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.317933627s
Jan 24 06:49:00.275: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:00.433: INFO: Pod "netserver-1": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.476415281s
Jan 24 06:49:00.433: INFO: The phase of Pod netserver-1 is Pending, waiting for it to be Running (with Ready = true)
Jan 24 06:49:00.434: INFO: Unexpected error:
<*pod.timeoutError | 0xc0047828a0>: {
msg: "timed out while waiting for pod nettest-3667/netserver-1 to be running and ready",
observedObjects: [
<*v1.Pod | 0xc00487b680>{
TypeMeta: {Kind: "", APIVersion: ""},
ObjectMeta: {
Name: "netserver-1",
GenerateName: "",
Namespace: "nettest-3667",
SelfLink: "",
UID: "0ba83d7f-3dea-415d-980f-2dc345b7995b",
ResourceVersion: "31151",
Generation: 0,
CreationTimestamp: {
Time: {
wall: 0,
ext: 63810139419,
loc: {
name: "Local",
zone: [
{name: "UTC", offset: 0, isDST: false},
],
tx: [
{
when: -9223372036854775808,
index: 0,
isstd: false,
isutc: false,
},
],
extend: "UTC0",
cacheStart: 9223372036854775807,
cacheEnd: 9223372036854775807,
cacheZone: {name: "UTC", offset: 0, isDST: false},
},
},
},
DeletionTimestamp: nil,
DeletionGracePeriodSeconds: nil,
Labels: {
"selector-b19e7ed4-0f0a-448d-8d84-7f33e98dae4f": "true",
},
Annotations: nil,
OwnerReferences: nil,
Finalizers: nil,
ManagedFields: [
{
Manager: "e2e.test",
Operation: "Update",
APIVersion: "v1",
Time: {
Time: {
wall: 0,
ext: 63810139419,
loc: {
name: "Local",
zone: [...],
tx: [...],
extend: "UTC0",
cacheStart: 9223372036854775807,
cacheEnd: 9223372036854775807,
cacheZone: {name: ..., offset: ..., isDST: ...},
},
},
},
FieldsType: "FieldsV1",
FieldsV1: {
Raw: "{\"f:metadata\":{\"f:labels\":{\".\":{},\"f:selector-b19e7ed4-0f0a-448d-8d84-7f33e98dae4f\":{}}},\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"webserver\\\"}\":{\".\":{},\"f:args\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:livenessProbe\":{\".\":{},\"f:failureThreshold\":{},\"f:httpGet\":{\".\":{},\"f:path\":{},\"f:port\":{},\"f:scheme\":{}},\"f:initialDelaySeconds\":{},\"f:periodSeconds\":{},\"f:successThreshold\":{},\"f:timeoutSeconds\":{}},\"f:name\":{},\"f:ports\":{\".\":{},\"k:{\\\"containerPort\\\":8081,\\\"protocol\\\":\\\"UDP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:name\":{},\"f:protocol\":{}},\"k:{\\\"containerPort\\\":8083,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:name\":{},\"f:protocol\":{}}},\"f:readinessProbe\":{\".\":{},\"f:failureThreshold\":{},\"f:httpGet\":{\".\":{},\"f:path\":{},\"f:port\":{},\"f:scheme\":{}},\"f:initialDelaySeconds\":{},\"f:periodSeconds\":{},\"f:successThreshold\":{},\"f:timeoutSeconds\":{}},\"f:resources\":{},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:nodeSelector\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:terminationGracePeriodSeconds\":{}}}",
},
Subresource: "",
},
{
Manager: "kubelet",
Operation: "Update",
APIVersion: "v1",
Time: {
Time: {
wall: 0,
ext: 63810139419,
loc: {
name: "Local",
zone: [...],
tx: [...],
extend: "UTC0",
cacheStart: 9223372036854775807,
cacheEnd: 9223372036854775807,
cacheZone: {name: ..., offset: ..., isDST: ...},
},
},
},
FieldsType: "FieldsV1",
FieldsV1: {
Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}",
},
Subresource: "status",
},
],
},
Spec: {
Volumes: [
{
Name: "kube-api-access-d9sqq",
VolumeSource: {
HostPath: nil,
EmptyDir: nil,
GCEPersistentDisk: nil,
AWSElasticBlockStore: nil,
GitRepo: nil,
Secret: nil,
NFS: nil,
ISCSI: nil,
Glusterfs: nil,
PersistentVolumeClaim: nil,
RBD: nil,
FlexVolume: nil,
Cinder: nil,
CephFS: nil,
Flocker: nil,
DownwardAPI: nil,
FC: nil,
AzureFile: nil,
ConfigMap: nil,
VsphereVolume: nil,
Quobyte: nil,
AzureDisk: nil,
PhotonPersistentDisk: nil,
Projected: {
Sources: [
{
Secret: ...,
DownwardAPI: ...,
ConfigMap: ...,
ServiceAccountToken: ...,
},
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
Jan 24 06:49:00.434: FAIL: timed out while waiting for pod nettest-3667/netserver-1 to be running and ready
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc0039cee00, {0x75cb852, 0x9}, 0xc00393d650)
test/e2e/framework/network/utils.go:866 +0x1d0
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc0039cee00, 0x7fe0d847fc10?)
test/e2e/framework/network/utils.go:763 +0x55
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc0039cee00, 0x3e?)
test/e2e/framework/network/utils.go:778 +0x3e
k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc000e7d590, {0x0, 0x0, 0x7fe101cb5f18?})
test/e2e/framework/network/utils.go:131 +0x125
k8s.io/kubernetes/test/e2e/network.glob..func22.6.8()
test/e2e/network/networking.go:251 +0x36
[AfterEach] [sig-network] Networking
test/e2e/framework/node/init/init.go:32
Jan 24 06:49:00.435: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-network] Networking
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-network] Networking
dump namespaces | framework.go:196
�[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/24/23 06:49:00.594�[0m
�[1mSTEP:�[0m Collecting events from namespace "nettest-3667". �[38;5;243m01/24/23 06:49:00.594�[0m
�[1mSTEP:�[0m Found 32 events. �[38;5;243m01/24/23 06:49:00.753�[0m
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:43:38 +0000 UTC - event for netserver-0: {default-scheduler } Scheduled: Successfully assigned nettest-3667/netserver-0 to i-036f641c68d5ad392
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:43:39 +0000 UTC - event for netserver-0: {kubelet i-036f641c68d5ad392} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:43:39 +0000 UTC - event for netserver-0: {kubelet i-036f641c68d5ad392} Created: Created container webserver
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:43:39 +0000 UTC - event for netserver-1: {default-scheduler } Scheduled: Successfully assigned nettest-3667/netserver-1 to i-0697a70b9aa95b7c0
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:43:39 +0000 UTC - event for netserver-1: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5f18d0256155e9f1c9eabdef886349141e3cd66a57a076b9ba90aa1bbe1028d0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:43:39 +0000 UTC - event for netserver-2: {default-scheduler } Scheduled: Successfully assigned nettest-3667/netserver-2 to i-0e0a289a9d5c333a0
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:43:39 +0000 UTC - event for netserver-2: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4881c8db60cd295d7cb0fefec9735bde58e522486972cdecf7298140463c50eb": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:43:39 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cb6c773f4012409acd38b030950e23e8fae9249a714947fb3721c9e3f8158e02": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:43:39 +0000 UTC - event for netserver-3: {default-scheduler } Scheduled: Successfully assigned nettest-3667/netserver-3 to i-0f5ee4c33b6799e76
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:43:40 +0000 UTC - event for netserver-0: {kubelet i-036f641c68d5ad392} Started: Started container webserver
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:43:51 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e772c81ba4b9608de1bb7a605acc36935d54340d7f5775af63df611f2cbe0de0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:43:52 +0000 UTC - event for netserver-1: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "831e550db2cd25d9b60f6d18ea9f7f5b94f38ea4f06084032d250c56fddedf23": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:43:53 +0000 UTC - event for netserver-2: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "69a3d0a2f387e48148cad2f417c03f3225e3f9977fd27279a78d61d4ee9b982f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:44:03 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "deb9a16a5dbfa1385debef22119a65fcca0324df066d3d8f79727ed7b56f2999": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:44:06 +0000 UTC - event for netserver-1: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "befb480975892d096dee351c94f2122bd6f53c1fff92be8cf5f470a368dc6dae": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:44:06 +0000 UTC - event for netserver-2: {kubelet i-0e0a289a9d5c333a0} Started: Started container webserver
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:44:06 +0000 UTC - event for netserver-2: {kubelet i-0e0a289a9d5c333a0} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:44:06 +0000 UTC - event for netserver-2: {kubelet i-0e0a289a9d5c333a0} Created: Created container webserver
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:44:15 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "08aef43172c7da8b8246728d4695ba5c58bfa28e0a7c7ac0c5edd27908b2025c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:44:20 +0000 UTC - event for netserver-1: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "10e7f14a63357f15028a345bcbc4f372fb20044014d9beace204a1c160cd8bc0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:44:29 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "85db473a88ae0a8183feb19b42e5e9a634190eb3cb14668bd1118f34ca1a72a6": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:44:36 +0000 UTC - event for netserver-1: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e486176ddb4dd18233fa3b826ecde4808ad67e742de8b226f634f95580f64a14": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:44:45 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0065bef3f4127b212979e232232a6b93a59f29b22c8c26e735b139e99bfe74ac": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:44:51 +0000 UTC - event for netserver-1: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "6fe5d9c6a3f5c419242005eecd5694565b0753107421b23a717326a45eba13ad": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:44:58 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5da2562753597fe0309628ea6a4edfcd6ffc7b66a88b5d4d0889d6ae68249975": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:45:04 +0000 UTC - event for netserver-1: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "60797e436d86bbfa38eb97bc652de4ff799977a11c7dbe83caf767267c6355d2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:45:12 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ea783023b03ee61eae07a034e0c523d2d636ae7094d4d476bc5fe0ccaacd02a2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:45:19 +0000 UTC - event for netserver-1: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "76e618eaa2ec2490695b15c317d754e3044d9e592365295601a474f38c4ff1f9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:45:27 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "458b47840c52ff151cf6a9038673c95d5328589d20318c161243483baeee370d": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:45:34 +0000 UTC - event for netserver-1: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f7f82baa3944917288f298fdafb7c594603a3a7d215048f4fb963d49fa731b27": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:45:41 +0000 UTC - event for netserver-3: {kubelet i-0f5ee4c33b6799e76} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "7f2bed92663e11a5c78ac839bb51731800979a308e429362c97de634815ba7e3": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.753: INFO: At 2023-01-24 06:45:48 +0000 UTC - event for netserver-1: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "514f5f88cd8a11e720b48f7102abfa93471cb368d15b471b078bf81b3be10648": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available
Jan 24 06:49:00.912: INFO: POD NODE PHASE GRACE CONDITIONS
Jan 24 06:49:00.912: INFO: netserver-0 i-036f641c68d5ad392 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:38 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:59 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:59 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:38 +0000 UTC }]
Jan 24 06:49:00.912: INFO: netserver-1 i-0697a70b9aa95b7c0 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:39 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:39 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:39 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:39 +0000 UTC }]
Jan 24 06:49:00.912: INFO: netserver-2 i-0e0a289a9d5c333a0 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:39 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:44:19 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:44:19 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:39 +0000 UTC }]
Jan 24 06:49:00.912: INFO: netserver-3 i-0f5ee4c33b6799e76 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:39 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:39 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:39 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:43:39 +0000 UTC }]
Jan 24 06:49:00.912: INFO:
Jan 24 06:49:01.232: INFO: Unable to fetch nettest-3667/netserver-1/webserver logs: the server rejected our request for an unknown reason (get pods netserver-1)
Jan 24 06:49:01.552: INFO: Unable to fetch nettest-3667/netserver-3/webserver logs: the server rejected our request for an unknown reason (get pods netserver-3)
Jan 24 06:49:01.712: INFO:
Logging node info for node i-036f641c68d5ad392
Jan 24 06:49:01.870: INFO: Node Info: &Node{ObjectMeta:{i-036f641c68d5ad392 50721840-3696-4207-81a9-8cd089fbb539 33468 0 2023-01-24 06:27:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-036f641c68d5ad392 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-036f641c68d5ad392 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.254 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-8398":"i-036f641c68d5ad392","csi-hostpath-volume-expand-5409":"i-036f641c68d5ad392","ebs.csi.aws.com":"i-036f641c68d5ad392"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:48:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2023-01-24 06:48:57 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-036f641c68d5ad392,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:54 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:54 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:54 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:48:54 +0000 UTC,LastTransitionTime:2023-01-24 06:27:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.254,},NodeAddress{Type:ExternalIP,Address:15.165.237.223,},NodeAddress{Type:InternalDNS,Address:i-036f641c68d5ad392.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-036f641c68d5ad392.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-165-237-223.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2732c784a9bb7e64a787956c82cf85,SystemUUID:ec2732c7-84a9-bb7e-64a7-87956c82cf85,BootID:bee5ed40-42b1-475e-b72b-16d715de535a,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-0269089ae9a4173b7],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0269089ae9a4173b7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-5409^2c26217c-9bb3-11ed-bce5-86d6f2bc551f,DevicePath:,},},Config:nil,},}
Jan 24 06:49:01.870: INFO:
Logging kubelet events for node i-036f641c68d5ad392
Jan 24 06:49:02.032: INFO:
Logging pods the kubelet thinks is on node i-036f641c68d5ad392
Jan 24 06:49:02.199: INFO: pod-56c7bad9-2975-4685-aab0-0cfb8b42331c started at 2023-01-24 06:48:56 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.199: INFO: Container write-pod ready: false, restart count 0
Jan 24 06:49:02.199: INFO: rs-76gck started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.199: INFO: Container donothing ready: false, restart count 0
Jan 24 06:49:02.199: INFO: pfpod started at 2023-01-24 06:48:43 +0000 UTC (0+2 container statuses recorded)
Jan 24 06:49:02.199: INFO: Container portforwardtester ready: false, restart count 0
Jan 24 06:49:02.199: INFO: Container readiness ready: false, restart count 0
Jan 24 06:49:02.199: INFO: hostexec-i-036f641c68d5ad392-g49z8 started at 2023-01-24 06:47:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.199: INFO: Container agnhost-container ready: true, restart count 0
Jan 24 06:49:02.199: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6twph started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.199: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:02.199: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-m4sr7 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.199: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:02.199: INFO: csi-mockplugin-0 started at 2023-01-24 06:47:12 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:49:02.199: INFO: Container csi-provisioner ready: false, restart count 0
Jan 24 06:49:02.199: INFO: Container driver-registrar ready: false, restart count 0
Jan 24 06:49:02.199: INFO: Container mock ready: false, restart count 0
Jan 24 06:49:02.199: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2mkph started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.199: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:02.199: INFO: rs-6s4df started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.199: INFO: Container donothing ready: false, restart count 0
Jan 24 06:49:02.199: INFO: csi-hostpathplugin-0 started at 2023-01-24 06:44:58 +0000 UTC (0+7 container statuses recorded)
Jan 24 06:49:02.199: INFO: Container csi-attacher ready: true, restart count 0
Jan 24 06:49:02.199: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:49:02.199: INFO: Container csi-resizer ready: true, restart count 0
Jan 24 06:49:02.199: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 24 06:49:02.199: INFO: Container hostpath ready: true, restart count 0
Jan 24 06:49:02.199: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:49:02.199: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:49:02.200: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-8sfzb started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:49:02.200: INFO: rs-zmrj2 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container donothing ready: false, restart count 0
Jan 24 06:49:02.200: INFO: csi-hostpathplugin-0 started at 2023-01-24 06:42:31 +0000 UTC (0+7 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container csi-attacher ready: true, restart count 0
Jan 24 06:49:02.200: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:49:02.200: INFO: Container csi-resizer ready: true, restart count 0
Jan 24 06:49:02.200: INFO: Container csi-snapshotter ready: true, restart count 0
Jan 24 06:49:02.200: INFO: Container hostpath ready: true, restart count 0
Jan 24 06:49:02.200: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:49:02.200: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:49:02.200: INFO: pod-projected-configmaps-495807fd-45a6-4d3f-a257-d364553fb370 started at 2023-01-24 06:48:40 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container agnhost-container ready: false, restart count 0
Jan 24 06:49:02.200: INFO: service-headless-btk74 started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container service-headless ready: false, restart count 0
Jan 24 06:49:02.200: INFO: pod-da9584f9-0d79-4167-ad37-ca206c3ac911 started at 2023-01-24 06:43:12 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container write-pod ready: false, restart count 0
Jan 24 06:49:02.200: INFO: ebs-csi-node-6z65r started at 2023-01-24 06:27:40 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:49:02.200: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:49:02.200: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:49:02.200: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cpp7h started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:02.200: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6hnbl started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:02.200: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-h5mpm started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:02.200: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-tcp26 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:02.200: INFO: csi-mockplugin-resizer-0 started at 2023-01-24 06:47:12 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container csi-resizer ready: false, restart count 0
Jan 24 06:49:02.200: INFO: pod-subpath-test-preprovisionedpv-l2p7 started at 2023-01-24 06:48:02 +0000 UTC (1+2 container statuses recorded)
Jan 24 06:49:02.200: INFO: Init container test-init-subpath-preprovisionedpv-l2p7 ready: false, restart count 0
Jan 24 06:49:02.200: INFO: Container test-container-subpath-preprovisionedpv-l2p7 ready: false, restart count 0
Jan 24 06:49:02.200: INFO: Container test-container-volume-preprovisionedpv-l2p7 ready: false, restart count 0
Jan 24 06:49:02.200: INFO: pod-handle-http-request started at 2023-01-24 06:48:15 +0000 UTC (0+2 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container container-handle-http-request ready: false, restart count 0
Jan 24 06:49:02.200: INFO: Container container-handle-https-request ready: false, restart count 0
Jan 24 06:49:02.200: INFO: netserver-0 started at 2023-01-24 06:43:38 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container webserver ready: true, restart count 0
Jan 24 06:49:02.200: INFO: cilium-9ht2g started at 2023-01-24 06:27:40 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:49:02.200: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:49:02.200: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:49:02.200: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-t9b6f started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:02.200: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-lc7s4 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:02.200: INFO: rand-non-local-gxd9p started at 2023-01-24 06:42:31 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:02.200: INFO: Container c ready: false, restart count 0
Jan 24 06:49:02.898: INFO:
Latency metrics for node i-036f641c68d5ad392
Jan 24 06:49:02.898: INFO:
Logging node info for node i-0697a70b9aa95b7c0
Jan 24 06:49:03.056: INFO: Node Info: &Node{ObjectMeta:{i-0697a70b9aa95b7c0 d0a3c976-eb07-4323-b48b-ec4945de7410 33229 0 2023-01-24 06:27:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0697a70b9aa95b7c0 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0697a70b9aa95b7c0 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.153 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0697a70b9aa95b7c0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:48:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0697a70b9aa95b7c0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:48:42 +0000 UTC,LastTransitionTime:2023-01-24 06:27:49 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.153,},NodeAddress{Type:ExternalIP,Address:3.38.96.49,},NodeAddress{Type:InternalDNS,Address:i-0697a70b9aa95b7c0.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0697a70b9aa95b7c0.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-38-96-49.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f1e14d470f251c9d042c57478cb8b,SystemUUID:ec2f1e14-d470-f251-c9d0-42c57478cb8b,BootID:578a4c1d-084e-4aeb-92fe-f64671842d82,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:49:03.057: INFO:
Logging kubelet events for node i-0697a70b9aa95b7c0
Jan 24 06:49:03.218: INFO:
Logging pods the kubelet thinks is on node i-0697a70b9aa95b7c0
Jan 24 06:49:03.384: INFO: ebs-csi-node-pkrr4 started at 2023-01-24 06:27:36 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:49:03.384: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:49:03.384: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:49:03.384: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-fgc56 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:03.384: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2v7c5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:03.384: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-4sk4h started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:03.384: INFO: csi-mockplugin-0 started at 2023-01-24 06:48:54 +0000 UTC (0+4 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container busybox ready: false, restart count 0
Jan 24 06:49:03.384: INFO: Container csi-provisioner ready: false, restart count 0
Jan 24 06:49:03.384: INFO: Container driver-registrar ready: false, restart count 0
Jan 24 06:49:03.384: INFO: Container mock ready: false, restart count 0
Jan 24 06:49:03.384: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-68w92 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:03.384: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qphds started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:03.384: INFO: webserver-pod started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container agnhost ready: false, restart count 0
Jan 24 06:49:03.384: INFO: service-headless-xh4pl started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container service-headless ready: false, restart count 0
Jan 24 06:49:03.384: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qtzq5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:03.384: INFO: netserver-1 started at 2023-01-24 06:43:39 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container webserver ready: false, restart count 0
Jan 24 06:49:03.384: INFO: rs-b9vhw started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container donothing ready: false, restart count 0
Jan 24 06:49:03.384: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2tkvs started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:49:03.384: INFO: pod-subpath-test-inlinevolume-qdbt started at 2023-01-24 06:44:17 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Init container init-volume-inlinevolume-qdbt ready: false, restart count 0
Jan 24 06:49:03.384: INFO: Container test-container-subpath-inlinevolume-qdbt ready: false, restart count 0
Jan 24 06:49:03.384: INFO: csi-mockplugin-attacher-0 started at 2023-01-24 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container csi-attacher ready: false, restart count 0
Jan 24 06:49:03.384: INFO: cilium-dh7hm started at 2023-01-24 06:27:36 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:49:03.384: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:49:03.384: INFO: coredns-559769c974-6wblg started at 2023-01-24 06:27:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container coredns ready: true, restart count 0
Jan 24 06:49:03.384: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-5pcw5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:03.384: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cdkdr started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:03.384: INFO: rs-w9wjl started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container donothing ready: true, restart count 0
Jan 24 06:49:03.384: INFO: csi-mockplugin-0 started at 2023-01-24 06:44:33 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container csi-provisioner ready: false, restart count 0
Jan 24 06:49:03.384: INFO: Container driver-registrar ready: false, restart count 0
Jan 24 06:49:03.384: INFO: Container mock ready: false, restart count 0
Jan 24 06:49:03.384: INFO: csi-mockplugin-resizer-0 started at 2023-01-24 06:44:33 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container csi-resizer ready: false, restart count 0
Jan 24 06:49:03.384: INFO: coredns-autoscaler-7cb5c5b969-vktlf started at 2023-01-24 06:27:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container autoscaler ready: true, restart count 0
Jan 24 06:49:03.384: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-xbpd8 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:03.384: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:49:03.977: INFO:
Latency metrics for node i-0697a70b9aa95b7c0
Jan 24 06:49:03.977: INFO:
Logging node info for node i-0b0b3ac061164cea4
Jan 24 06:49:04.136: INFO: Node Info: &Node{ObjectMeta:{i-0b0b3ac061164cea4 ac00aaf7-4427-49f6-b849-9c812ae80069 33008 0 2023-01-24 06:25:23 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b0b3ac061164cea4 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.51.48 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b0b3ac061164cea4"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-24 06:25:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-24 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:26:22 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:26:22 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-24 06:26:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-24 06:48:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0b0b3ac061164cea4,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892297728 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787440128 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:48:11 +0000 UTC,LastTransitionTime:2023-01-24 06:26:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.48,},NodeAddress{Type:ExternalIP,Address:13.125.222.159,},NodeAddress{Type:InternalDNS,Address:i-0b0b3ac061164cea4.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b0b3ac061164cea4.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-125-222-159.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255807a7dec89d2cc9cfee6d481c20,SystemUUID:ec255807-a7de-c89d-2cc9-cfee6d481c20,BootID:4153aea0-5d07-4f37-a966-c56ef660acfc,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.1],SizeBytes:135178704,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.1],SizeBytes:124995897,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:63920476,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.1],SizeBytes:57661752,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:44339030,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42841305,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4965794,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:49:04.136: INFO:
Logging kubelet events for node i-0b0b3ac061164cea4
Jan 24 06:49:04.297: INFO:
Logging pods the kubelet thinks is on node i-0b0b3ac061164cea4
Jan 24 06:49:04.463: INFO: aws-cloud-controller-manager-4n9pv started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:04.463: INFO: Container aws-cloud-controller-manager ready: true, restart count 0
Jan 24 06:49:04.463: INFO: ebs-csi-node-q4jsj started at 2023-01-24 06:26:07 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:49:04.463: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:49:04.463: INFO: kops-controller-vfj77 started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:04.463: INFO: Container kops-controller ready: true, restart count 0
Jan 24 06:49:04.463: INFO: dns-controller-69987775c6-vzqc8 started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:04.463: INFO: Container dns-controller ready: true, restart count 0
Jan 24 06:49:04.463: INFO: ebs-csi-controller-5bd98b456f-kswjq started at 2023-01-24 06:26:07 +0000 UTC (0+5 container statuses recorded)
Jan 24 06:49:04.463: INFO: Container csi-attacher ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Container csi-resizer ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:49:04.463: INFO: kube-apiserver-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+2 container statuses recorded)
Jan 24 06:49:04.463: INFO: Container healthcheck ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Container kube-apiserver ready: true, restart count 3
Jan 24 06:49:04.463: INFO: etcd-manager-main-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (10+1 container statuses recorded)
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Container etcd-manager ready: true, restart count 0
Jan 24 06:49:04.463: INFO: kube-controller-manager-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:04.463: INFO: Container kube-controller-manager ready: true, restart count 4
Jan 24 06:49:04.463: INFO: kube-scheduler-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:04.463: INFO: Container kube-scheduler ready: true, restart count 0
Jan 24 06:49:04.463: INFO: cilium-w225f started at 2023-01-24 06:26:07 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:49:04.463: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:49:04.463: INFO: cilium-operator-5dd44dc49f-rn9bl started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:04.463: INFO: Container cilium-operator ready: true, restart count 0
Jan 24 06:49:04.463: INFO: etcd-manager-events-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (10+1 container statuses recorded)
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 24 06:49:04.463: INFO: Container etcd-manager ready: true, restart count 0
Jan 24 06:49:05.010: INFO:
Latency metrics for node i-0b0b3ac061164cea4
Jan 24 06:49:05.010: INFO:
Logging node info for node i-0e0a289a9d5c333a0
Jan 24 06:49:05.169: INFO: Node Info: &Node{ObjectMeta:{i-0e0a289a9d5c333a0 9194794b-d78a-4a30-bbec-b76df63f68f1 31709 0 2023-01-24 06:27:40 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0e0a289a9d5c333a0 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0e0a289a9d5c333a0 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.62 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0e0a289a9d5c333a0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:44:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0e0a289a9d5c333a0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:10 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:10 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:10 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:44:10 +0000 UTC,LastTransitionTime:2023-01-24 06:27:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.62,},NodeAddress{Type:ExternalIP,Address:54.180.158.26,},NodeAddress{Type:InternalDNS,Address:i-0e0a289a9d5c333a0.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0e0a289a9d5c333a0.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-180-158-26.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2281a7b00c77d27e8055a22bf9facf,SystemUUID:ec2281a7-b00c-77d2-7e80-55a22bf9facf,BootID:be2e68ba-6839-4b53-847f-22f29f0a7360,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:49:05.169: INFO:
Logging kubelet events for node i-0e0a289a9d5c333a0
Jan 24 06:49:05.337: INFO:
Logging pods the kubelet thinks is on node i-0e0a289a9d5c333a0
Jan 24 06:49:05.503: INFO: csi-mockplugin-attacher-0 started at 2023-01-24 06:48:19 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container csi-attacher ready: false, restart count 0
Jan 24 06:49:05.503: INFO: alpine-nnp-false-2c8bfdf9-3bfb-42e5-835c-fb8faa43651e started at 2023-01-24 06:46:40 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container alpine-nnp-false-2c8bfdf9-3bfb-42e5-835c-fb8faa43651e ready: false, restart count 0
Jan 24 06:49:05.503: INFO: hostexec-i-0e0a289a9d5c333a0-qhktd started at 2023-01-24 06:48:55 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container agnhost-container ready: true, restart count 0
Jan 24 06:49:05.503: INFO: dns-test-b3d6c428-d0cf-4d5a-9700-fdfb874b5d91 started at 2023-01-24 06:48:41 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container jessie-querier ready: false, restart count 0
Jan 24 06:49:05.503: INFO: Container querier ready: false, restart count 0
Jan 24 06:49:05.503: INFO: Container webserver ready: false, restart count 0
Jan 24 06:49:05.503: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-9hr5l started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:05.503: INFO: netserver-2 started at 2023-01-24 06:43:39 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container webserver ready: true, restart count 0
Jan 24 06:49:05.503: INFO: service-headless-h4z88 started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container service-headless ready: false, restart count 0
Jan 24 06:49:05.503: INFO: rs-l5fgd started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container donothing ready: false, restart count 0
Jan 24 06:49:05.503: INFO: cilium-cvt86 started at 2023-01-24 06:27:41 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:49:05.503: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:49:05.503: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-224qm started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:05.503: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-z9lc4 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:05.503: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-7nhf8 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:05.503: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-w7m2k started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0
Jan 24 06:49:05.503: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-867tg started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:05.503: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-zv4q6 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:05.503: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-tbbms started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:05.503: INFO: pod-75cd15d0-1655-4201-a505-413ddae34312 started at 2023-01-24 06:48:59 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container write-pod ready: false, restart count 0
Jan 24 06:49:05.503: INFO: ebs-csi-node-v6prc started at 2023-01-24 06:27:41 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:49:05.503: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:49:05.503: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:49:05.503: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-ts8xr started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:05.503: INFO: rs-rsqdt started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container donothing ready: true, restart count 0
Jan 24 06:49:05.503: INFO: rc-test-qrc82 started at 2023-01-24 06:48:11 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container rc-test ready: false, restart count 0
Jan 24 06:49:05.503: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-bkjzk started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:05.503: INFO: csi-mockplugin-0 started at 2023-01-24 06:48:18 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container csi-provisioner ready: false, restart count 0
Jan 24 06:49:05.503: INFO: Container driver-registrar ready: false, restart count 0
Jan 24 06:49:05.503: INFO: Container mock ready: false, restart count 0
Jan 24 06:49:05.503: INFO: rs-znpr8 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container donothing ready: false, restart count 0
Jan 24 06:49:05.503: INFO: startup-e83c2b2e-e8d4-4137-b286-5a3397f52961 started at 2023-01-24 06:49:02 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:05.503: INFO: Container busybox ready: false, restart count 0
Jan 24 06:49:06.077: INFO:
Latency metrics for node i-0e0a289a9d5c333a0
Jan 24 06:49:06.077: INFO:
Logging node info for node i-0f5ee4c33b6799e76
Jan 24 06:49:06.236: INFO: Node Info: &Node{ObjectMeta:{i-0f5ee4c33b6799e76 34c5957d-82ce-41f2-824d-90a78fa8bbba 31698 0 2023-01-24 06:27:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0f5ee4c33b6799e76 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0f5ee4c33b6799e76 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.57.132 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-7943":"i-0f5ee4c33b6799e76","ebs.csi.aws.com":"i-0f5ee4c33b6799e76"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:44:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0f5ee4c33b6799e76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:19 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:19 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:44:19 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:44:19 +0000 UTC,LastTransitionTime:2023-01-24 06:27:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.57.132,},NodeAddress{Type:ExternalIP,Address:3.36.124.131,},NodeAddress{Type:InternalDNS,Address:i-0f5ee4c33b6799e76.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f5ee4c33b6799e76.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-36-124-131.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bcf8e071d2b94271b14b41c978c,SystemUUID:ec2f5bcf-8e07-1d2b-9427-1b14b41c978c,BootID:400ceb00-7274-4349-89f8-a3cb13aed9c2,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 24 06:49:06.237: INFO:
Logging kubelet events for node i-0f5ee4c33b6799e76
Jan 24 06:49:06.398: INFO:
Logging pods the kubelet thinks is on node i-0f5ee4c33b6799e76
Jan 24 06:49:06.563: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6lkqq started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:06.563: INFO: hostexec-i-0f5ee4c33b6799e76-26jmv started at 2023-01-24 06:43:55 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container agnhost-container ready: true, restart count 0
Jan 24 06:49:06.563: INFO: pod-failure-failjob-s2mc2 started at 2023-01-24 06:49:00 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container c ready: false, restart count 0
Jan 24 06:49:06.563: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qmz8q started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:06.563: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-xl6bg started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:06.563: INFO: rs-g6rhr started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container donothing ready: false, restart count 0
Jan 24 06:49:06.563: INFO: rs-8b8n9 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container donothing ready: false, restart count 0
Jan 24 06:49:06.563: INFO: pvc-volume-tester-slzmq started at 2023-01-24 06:44:20 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container volume-tester ready: false, restart count 0
Jan 24 06:49:06.563: INFO: cilium-ql489 started at 2023-01-24 06:27:40 +0000 UTC (1+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Init container clean-cilium-state ready: true, restart count 0
Jan 24 06:49:06.563: INFO: Container cilium-agent ready: true, restart count 0
Jan 24 06:49:06.563: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-hwktw started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:06.563: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cfnfh started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:06.563: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-7lw99 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:06.563: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-w9729 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:06.563: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-lvvw2 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:06.563: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-wjt45 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:06.563: INFO: coredns-559769c974-5gltc started at 2023-01-24 06:28:16 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container coredns ready: true, restart count 0
Jan 24 06:49:06.563: INFO: csi-mockplugin-0 started at 2023-01-24 06:43:09 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container csi-provisioner ready: true, restart count 0
Jan 24 06:49:06.563: INFO: Container driver-registrar ready: true, restart count 0
Jan 24 06:49:06.563: INFO: Container mock ready: true, restart count 0
Jan 24 06:49:06.563: INFO: netserver-3 started at 2023-01-24 06:43:39 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container webserver ready: false, restart count 0
Jan 24 06:49:06.563: INFO: ebs-csi-node-vjhdb started at 2023-01-24 06:27:40 +0000 UTC (0+3 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container ebs-plugin ready: true, restart count 0
Jan 24 06:49:06.563: INFO: Container liveness-probe ready: true, restart count 0
Jan 24 06:49:06.563: INFO: Container node-driver-registrar ready: true, restart count 0
Jan 24 06:49:06.563: INFO: pod-failure-failjob-s22zv started at 2023-01-24 06:49:00 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container c ready: false, restart count 0
Jan 24 06:49:06.563: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-r5vgx started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded)
Jan 24 06:49:06.563: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0
Jan 24 06:49:07.127: INFO:
Latency metrics for node i-0f5ee4c33b6799e76
[DeferCleanup (Each)] [sig-network] Networking
tear down framework | framework.go:193
�[1mSTEP:�[0m Destroying namespace "nettest-3667" for this suite. �[38;5;243m01/24/23 06:49:07.127�[0m
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sServices\sshould\simplement\sservice\.kubernetes\.io\/headless$'
test/e2e/network/service.go:2322 k8s.io/kubernetes/test/e2e/network.glob..func26.32() test/e2e/network/service.go:2322 +0x225from junit_01.xml
[BeforeEach] [sig-network] Services set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/24/23 06:44:47.753�[0m Jan 24 06:44:47.753: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename services �[38;5;243m01/24/23 06:44:47.754�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/24/23 06:44:48.219�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/24/23 06:44:48.527�[0m [BeforeEach] [sig-network] Services test/e2e/framework/metrics/init/init.go:31 [BeforeEach] [sig-network] Services test/e2e/network/service.go:766 [It] should implement service.kubernetes.io/headless test/e2e/network/service.go:2307 �[1mSTEP:�[0m creating service-headless in namespace services-6896 �[38;5;243m01/24/23 06:44:48.835�[0m �[1mSTEP:�[0m creating service service-headless in namespace services-6896 �[38;5;243m01/24/23 06:44:48.835�[0m �[1mSTEP:�[0m creating replication controller service-headless in namespace services-6896 �[38;5;243m01/24/23 06:44:48.995�[0m I0124 06:44:49.152490 6690 runners.go:193] Created replication controller with name: service-headless, namespace: services-6896, replica count: 3 I0124 06:44:52.354328 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:44:55.355198 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:44:58.355419 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:01.356172 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:04.357373 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:07.358135 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:10.358301 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:13.359417 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:16.360004 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:19.360677 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:22.361307 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:25.361827 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:28.362515 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:31.363339 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:34.363583 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:37.364017 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:40.364119 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:43.364581 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:46.364810 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:49.365027 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:52.365478 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:55.366160 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:45:58.366359 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:01.366864 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:04.367804 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:07.368248 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:10.368442 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:13.368646 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:16.368888 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:19.369286 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:22.369453 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:25.369679 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:28.370380 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:31.370773 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:34.371397 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:37.372323 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:40.373221 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:43.373925 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:46.374176 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:49.375336 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:52.375748 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:55.376421 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:46:58.377428 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:01.378259 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:04.378408 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:07.378619 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:10.379110 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:13.380081 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:16.380319 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:19.381243 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:22.381366 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:25.382280 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:28.383351 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:31.384154 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:34.384858 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:37.385576 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:40.386382 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:43.387144 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:46.388280 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:49.389268 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:52.389960 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:55.390152 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:47:58.390483 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:01.391334 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:04.392390 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:07.393182 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:10.393868 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:13.394061 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:16.395118 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:19.395911 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:22.396711 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:25.396978 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:28.397626 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:31.398445 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:34.399336 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:37.400070 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:40.400432 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:43.400601 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:46.401232 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:49.401392 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:52.402527 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:55.403434 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:48:58.404515 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:01.405281 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:04.406414 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 0 running, 3 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:07.406735 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:10.407135 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:13.407947 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:16.409082 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:19.409775 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:22.410487 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:25.410726 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:28.411791 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:31.412174 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:34.412485 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:37.413342 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:40.414229 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:43.414890 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:46.415042 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:49.415951 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:52.416357 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:55.417180 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:49:58.418288 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:01.419349 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:04.420092 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:07.420253 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:10.421043 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:13.421843 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:16.422134 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:19.422892 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:22.423400 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:25.424382 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:28.425356 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:31.426102 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:34.427138 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:37.428292 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:40.429238 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:43.429509 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:46.430176 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:49.430404 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:52.431492 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:55.432236 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:50:58.433115 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:01.433276 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:04.434025 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:07.434178 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:10.434464 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:13.435229 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:16.436142 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:19.436289 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:22.436571 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:25.437063 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:28.437434 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:31.438097 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:34.438355 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:37.438473 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:40.439301 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:43.440363 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:46.441163 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:49.441395 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:52.441584 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:55.442425 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:51:58.443104 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:01.443249 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:04.443927 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:07.444141 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:10.444892 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:13.445873 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:16.446900 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:19.447398 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:22.447631 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:25.448426 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:28.449480 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:31.449755 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:34.450016 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:37.450276 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:40.451425 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:43.451692 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:46.452431 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:49.453479 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:52.454575 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:55.455269 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:52:58.456107 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:01.456326 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:04.457328 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:07.458522 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:10.459405 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:13.459768 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:16.459985 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:19.460888 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:22.461561 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:25.462339 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:28.462735 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:31.463116 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:34.463349 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:37.464150 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:40.465316 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:43.466397 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:46.467106 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:49.467370 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:52.468487 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:55.469248 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:53:58.469517 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:54:01.470300 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:54:04.470454 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:54:07.471415 6690 runners.go:193] service-headless Pods: 3 out of 3 created, 1 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0124 06:54:07.626640 6690 runners.go:193] Pod service-headless-btk74 i-036f641c68d5ad392 Pending <nil> I0124 06:54:07.626689 6690 runners.go:193] Pod service-headless-h4z88 i-0e0a289a9d5c333a0 Running <nil> I0124 06:54:07.626698 6690 runners.go:193] Pod service-headless-xh4pl i-0697a70b9aa95b7c0 Pending <nil> Jan 24 06:54:07.626: INFO: Unexpected error: failed to create replication controller with headless service: in the namespace: services-6896: <*errors.errorString | 0xc006f56240>: { s: "only 1 pods started out of 3", } Jan 24 06:54:07.626: FAIL: failed to create replication controller with headless service: in the namespace: services-6896: only 1 pods started out of 3 Full Stack Trace k8s.io/kubernetes/test/e2e/network.glob..func26.32() test/e2e/network/service.go:2322 +0x225 [AfterEach] [sig-network] Services test/e2e/framework/node/init/init.go:32 Jan 24 06:54:07.627: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-network] Services test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-network] Services dump namespaces | framework.go:196 �[1mSTEP:�[0m dump namespace information after failure �[38;5;243m01/24/23 06:54:07.782�[0m �[1mSTEP:�[0m Collecting events from namespace "services-6896". �[38;5;243m01/24/23 06:54:07.782�[0m �[1mSTEP:�[0m Found 40 events. �[38;5;243m01/24/23 06:54:07.937�[0m Jan 24 06:54:07.937: INFO: At 2023-01-24 06:44:49 +0000 UTC - event for service-headless: {replication-controller } SuccessfulCreate: Created pod: service-headless-btk74 Jan 24 06:54:07.937: INFO: At 2023-01-24 06:44:49 +0000 UTC - event for service-headless: {endpoint-controller } FailedToUpdateEndpoint: Failed to update endpoint services-6896/service-headless: Operation cannot be fulfilled on endpoints "service-headless": the object has been modified; please apply your changes to the latest version and try again Jan 24 06:54:07.937: INFO: At 2023-01-24 06:44:49 +0000 UTC - event for service-headless: {replication-controller } SuccessfulCreate: Created pod: service-headless-xh4pl Jan 24 06:54:07.937: INFO: At 2023-01-24 06:44:49 +0000 UTC - event for service-headless: {replication-controller } SuccessfulCreate: Created pod: service-headless-h4z88 Jan 24 06:54:07.937: INFO: At 2023-01-24 06:44:49 +0000 UTC - event for service-headless-btk74: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "fc0324250a8210e15489168eff64cfde46a7ce2a7a85b008d4cef9348a79f7db": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:44:49 +0000 UTC - event for service-headless-btk74: {default-scheduler } Scheduled: Successfully assigned services-6896/service-headless-btk74 to i-036f641c68d5ad392 Jan 24 06:54:07.937: INFO: At 2023-01-24 06:44:49 +0000 UTC - event for service-headless-h4z88: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "dd6aa39c3c9e811758c8a8dea19244f7ec99c28a70fdcf0918a3544af48fdacf": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:44:49 +0000 UTC - event for service-headless-h4z88: {default-scheduler } Scheduled: Successfully assigned services-6896/service-headless-h4z88 to i-0e0a289a9d5c333a0 Jan 24 06:54:07.937: INFO: At 2023-01-24 06:44:49 +0000 UTC - event for service-headless-xh4pl: {default-scheduler } Scheduled: Successfully assigned services-6896/service-headless-xh4pl to i-0697a70b9aa95b7c0 Jan 24 06:54:07.937: INFO: At 2023-01-24 06:44:49 +0000 UTC - event for service-headless-xh4pl: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "39c661e92f3cd3ae3648cbd51c2f2125e3fff895b938588d458db91e640d8905": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:45:00 +0000 UTC - event for service-headless-btk74: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "36ffb5ff44d64860ed29e948bba9ddac817a586246d0259f0021461ba84bee38": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:45:02 +0000 UTC - event for service-headless-xh4pl: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e9ebe62f6521eefcf7d935462e507bfb1e55bcbf715378bbc5a0e6f7ca5198cf": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:45:04 +0000 UTC - event for service-headless-h4z88: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "446dc368219f15b51d86bd5971bf22cf7a08f042154fabf497e58b826e402f69": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:45:14 +0000 UTC - event for service-headless-btk74: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "fce9919f74bd11f70a3d148bc83d631d2376c050ac73fd654e839b7ba72451a5": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:45:15 +0000 UTC - event for service-headless-xh4pl: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bde6a0b7b479b13651d2dd14989c85f52681041b544ffed6ce7d2af0858520f8": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:45:16 +0000 UTC - event for service-headless-h4z88: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "178a54545dc14e559b3acf2a0119abc8d5503bc4602fd22f8c68f8fa9a4b8c62": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:45:25 +0000 UTC - event for service-headless-btk74: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "49ca0b352400a7a24d517f2a2304fd37db7af116ea9bfcfcc4f0e6fbed7ff844": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:45:28 +0000 UTC - event for service-headless-h4z88: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "03a301372e2716788d9450270820087a0b00cce8a418b5d066e469b0e826bcb0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:45:29 +0000 UTC - event for service-headless-xh4pl: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c75b738917658e84a02a3ba488c47337a18662e37c8dc1973f3a4d69417760cc": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:45:37 +0000 UTC - event for service-headless-btk74: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "45f6a65c9d1116d6e7fb1c1894e6dc0a9bc69b6936668339a10ad7488ede3d5f": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:45:40 +0000 UTC - event for service-headless-xh4pl: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b052e466b44e14b9899fba7128aedee109356475261cfeac2348298bdbb782ff": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:45:42 +0000 UTC - event for service-headless-h4z88: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c14e618d9e8d9382f84d213d067292c5ff6c14933ec8867cc88939fd3a64748e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:45:49 +0000 UTC - event for service-headless-btk74: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "37296cf33560f28222d33eede7edc11fcb9ce3efc1f97fa8424642c7a37d10e2": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:45:52 +0000 UTC - event for service-headless-xh4pl: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "69f93bf7ee898f302cc2069c3a54ba983a8334cf257b22ae6c4aba076ab4d87c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:45:55 +0000 UTC - event for service-headless-h4z88: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c8dfbe76c8d2f3265d2b10f2ccdc91dc920e943b2be4c22e8b5c6dc72421d74b": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:46:00 +0000 UTC - event for service-headless-btk74: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d4b9f486d47842f465e1fdf51a8e5401da802720978924e6b65c92ed7b18abb9": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:46:07 +0000 UTC - event for service-headless-h4z88: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "29ffa74ef0216d4b7c9d5fecf16e95999a9a8c1fab2a2b6ad43090e34eb23f0c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:46:07 +0000 UTC - event for service-headless-xh4pl: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c32b85022d402b4423753112476bcb1f8852606b20baa96cc99af85844cbf813": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:46:15 +0000 UTC - event for service-headless-btk74: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9be2b160ac5f8ef19d2c6b540b182c16641f8a865ed127c2d0b9c08ba8f9abff": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:46:22 +0000 UTC - event for service-headless-h4z88: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5d6cc3595f5b956bcb7dd217ef9d048c52ded3d83e923bcc7694b62272ea89ab": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:46:23 +0000 UTC - event for service-headless-xh4pl: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b915311c8ce9d28b0f3c9fede9adc9c37bffd088af5a2859761d1594cdcae7ab": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:46:27 +0000 UTC - event for service-headless-btk74: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "2bcab88affe245968ee1f5c901129ebbdb28ecb6ba6dce344076c72894be4a5c": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:46:34 +0000 UTC - event for service-headless-h4z88: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "dbe12fd9c8284415b1c498492ca135dfb1a5de59bd50379f930c543c1515cc1e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:46:34 +0000 UTC - event for service-headless-xh4pl: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "2cb05bc961b58fb6ad54c69389065560ac039bdc4e50478785d4cde768e4301e": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:46:42 +0000 UTC - event for service-headless-btk74: {kubelet i-036f641c68d5ad392} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1647f68eeb770134215fca33a17436c519d2aadd258ed97a183e229eab0da4b5": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.937: INFO: At 2023-01-24 06:46:45 +0000 UTC - event for service-headless-h4z88: {kubelet i-0e0a289a9d5c333a0} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5c3457da17d9e7e26512890382a04f77278d097535b57a3a2602f0ba7f7f35a0": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.938: INFO: At 2023-01-24 06:46:46 +0000 UTC - event for service-headless-xh4pl: {kubelet i-0697a70b9aa95b7c0} FailedCreatePodSandBox: (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f342e08d05843c18e71b2e794e86a3e49fa8e8523bbecf8bbe2353edd8eda862": plugin type="cilium-cni" name="cilium" failed (add): unable to allocate IP via local cilium agent: [POST /ipam][502] postIpamFailure No more IPs available Jan 24 06:54:07.938: INFO: At 2023-01-24 06:49:04 +0000 UTC - event for service-headless-h4z88: {kubelet i-0e0a289a9d5c333a0} Created: Created container service-headless Jan 24 06:54:07.938: INFO: At 2023-01-24 06:49:04 +0000 UTC - event for service-headless-h4z88: {kubelet i-0e0a289a9d5c333a0} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Jan 24 06:54:07.938: INFO: At 2023-01-24 06:49:05 +0000 UTC - event for service-headless-h4z88: {kubelet i-0e0a289a9d5c333a0} Started: Started container service-headless Jan 24 06:54:08.092: INFO: POD NODE PHASE GRACE CONDITIONS Jan 24 06:54:08.092: INFO: service-headless-btk74 i-036f641c68d5ad392 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:44:49 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:44:49 +0000 UTC ContainersNotReady containers with unready status: [service-headless]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:44:49 +0000 UTC ContainersNotReady containers with unready status: [service-headless]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:44:49 +0000 UTC }] Jan 24 06:54:08.092: INFO: service-headless-h4z88 i-0e0a289a9d5c333a0 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:44:49 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:49:05 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:49:05 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:44:49 +0000 UTC }] Jan 24 06:54:08.092: INFO: service-headless-xh4pl i-0697a70b9aa95b7c0 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:44:49 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:44:49 +0000 UTC ContainersNotReady containers with unready status: [service-headless]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:44:49 +0000 UTC ContainersNotReady containers with unready status: [service-headless]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-24 06:44:49 +0000 UTC }] Jan 24 06:54:08.092: INFO: Jan 24 06:54:08.248: INFO: Unable to fetch services-6896/service-headless-btk74/service-headless logs: the server rejected our request for an unknown reason (get pods service-headless-btk74) Jan 24 06:54:08.560: INFO: Unable to fetch services-6896/service-headless-xh4pl/service-headless logs: the server rejected our request for an unknown reason (get pods service-headless-xh4pl) Jan 24 06:54:08.715: INFO: Logging node info for node i-036f641c68d5ad392 Jan 24 06:54:08.870: INFO: Node Info: &Node{ObjectMeta:{i-036f641c68d5ad392 50721840-3696-4207-81a9-8cd089fbb539 35191 0 2023-01-24 06:27:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-036f641c68d5ad392 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-036f641c68d5ad392 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.53.254 csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-5409":"i-036f641c68d5ad392","ebs.csi.aws.com":"i-036f641c68d5ad392"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:54:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-036f641c68d5ad392,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:54:00 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:54:00 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:54:00 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:54:00 +0000 UTC,LastTransitionTime:2023-01-24 06:27:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.53.254,},NodeAddress{Type:ExternalIP,Address:15.165.237.223,},NodeAddress{Type:InternalDNS,Address:i-036f641c68d5ad392.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-036f641c68d5ad392.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-15-165-237-223.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2732c784a9bb7e64a787956c82cf85,SystemUUID:ec2732c7-84a9-bb7e-64a7-87956c82cf85,BootID:bee5ed40-42b1-475e-b72b-16d715de535a,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 24 06:54:08.870: INFO: Logging kubelet events for node i-036f641c68d5ad392 Jan 24 06:54:09.027: INFO: Logging pods the kubelet thinks is on node i-036f641c68d5ad392 Jan 24 06:54:09.190: INFO: dns-test-93a0bf56-7463-4d4f-a334-bb3e44707aaa started at 2023-01-24 06:51:48 +0000 UTC (0+3 container statuses recorded) Jan 24 06:54:09.190: INFO: Container jessie-querier ready: false, restart count 0 Jan 24 06:54:09.190: INFO: Container querier ready: false, restart count 0 Jan 24 06:54:09.190: INFO: Container webserver ready: false, restart count 0 Jan 24 06:54:09.190: INFO: rs-76gck started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container donothing ready: false, restart count 0 Jan 24 06:54:09.190: INFO: pod-subpath-test-downwardapi-jp8w started at 2023-01-24 06:53:24 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container test-container-subpath-downwardapi-jp8w ready: false, restart count 0 Jan 24 06:54:09.190: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6twph started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:09.190: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-m4sr7 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:09.190: INFO: csi-hostpathplugin-0 started at 2023-01-24 06:44:58 +0000 UTC (0+7 container statuses recorded) Jan 24 06:54:09.190: INFO: Container csi-attacher ready: true, restart count 0 Jan 24 06:54:09.190: INFO: Container csi-provisioner ready: true, restart count 0 Jan 24 06:54:09.190: INFO: Container csi-resizer ready: true, restart count 0 Jan 24 06:54:09.190: INFO: Container csi-snapshotter ready: true, restart count 0 Jan 24 06:54:09.190: INFO: Container hostpath ready: true, restart count 0 Jan 24 06:54:09.190: INFO: Container liveness-probe ready: true, restart count 0 Jan 24 06:54:09.190: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 24 06:54:09.190: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2mkph started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:09.190: INFO: rs-6s4df started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container donothing ready: false, restart count 0 Jan 24 06:54:09.190: INFO: pod-0 started at 2023-01-24 06:53:19 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container donothing ready: false, restart count 0 Jan 24 06:54:09.190: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-8sfzb started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:09.190: INFO: rs-zmrj2 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container donothing ready: false, restart count 0 Jan 24 06:54:09.190: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6hnbl started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:09.190: INFO: test-pod started at 2023-01-24 06:53:49 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container webserver ready: false, restart count 0 Jan 24 06:54:09.190: INFO: service-headless-btk74 started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container service-headless ready: false, restart count 0 Jan 24 06:54:09.190: INFO: ebs-csi-node-6z65r started at 2023-01-24 06:27:40 +0000 UTC (0+3 container statuses recorded) Jan 24 06:54:09.190: INFO: Container ebs-plugin ready: true, restart count 0 Jan 24 06:54:09.190: INFO: Container liveness-probe ready: true, restart count 0 Jan 24 06:54:09.190: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 24 06:54:09.190: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cpp7h started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:09.190: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-h5mpm started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:09.190: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-tcp26 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:09.190: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-lc7s4 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:09.190: INFO: rand-non-local-gxd9p started at 2023-01-24 06:42:31 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container c ready: false, restart count 0 Jan 24 06:54:09.190: INFO: cilium-9ht2g started at 2023-01-24 06:27:40 +0000 UTC (1+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 24 06:54:09.190: INFO: Container cilium-agent ready: true, restart count 0 Jan 24 06:54:09.190: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-t9b6f started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:09.190: INFO: test-ss-0 started at 2023-01-24 06:49:23 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container webserver ready: false, restart count 0 Jan 24 06:54:09.190: INFO: pod-configmaps-5f55ab46-5e11-4c96-87a9-f170ab7204ce started at 2023-01-24 06:49:28 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container agnhost-container ready: false, restart count 0 Jan 24 06:54:09.190: INFO: explicit-nonroot-uid started at 2023-01-24 06:53:27 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container explicit-nonroot-uid ready: false, restart count 0 Jan 24 06:54:09.190: INFO: server started at 2023-01-24 06:49:10 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:09.190: INFO: Container agnhost-container ready: true, restart count 0 Jan 24 06:54:09.821: INFO: Latency metrics for node i-036f641c68d5ad392 Jan 24 06:54:09.821: INFO: Logging node info for node i-0697a70b9aa95b7c0 Jan 24 06:54:09.976: INFO: Node Info: &Node{ObjectMeta:{i-0697a70b9aa95b7c0 d0a3c976-eb07-4323-b48b-ec4945de7410 35104 0 2023-01-24 06:27:35 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0697a70b9aa95b7c0 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0697a70b9aa95b7c0 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.153 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0697a70b9aa95b7c0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kubelet Update v1 2023-01-24 06:27:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:53:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0697a70b9aa95b7c0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:53:46 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:53:46 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:53:46 +0000 UTC,LastTransitionTime:2023-01-24 06:27:35 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:53:46 +0000 UTC,LastTransitionTime:2023-01-24 06:27:49 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.153,},NodeAddress{Type:ExternalIP,Address:3.38.96.49,},NodeAddress{Type:InternalDNS,Address:i-0697a70b9aa95b7c0.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0697a70b9aa95b7c0.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-38-96-49.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f1e14d470f251c9d042c57478cb8b,SystemUUID:ec2f1e14-d470-f251-c9d0-42c57478cb8b,BootID:578a4c1d-084e-4aeb-92fe-f64671842d82,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:20293261,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 24 06:54:09.977: INFO: Logging kubelet events for node i-0697a70b9aa95b7c0 Jan 24 06:54:10.134: INFO: Logging pods the kubelet thinks is on node i-0697a70b9aa95b7c0 Jan 24 06:54:10.297: INFO: ebs-csi-node-pkrr4 started at 2023-01-24 06:27:36 +0000 UTC (0+3 container statuses recorded) Jan 24 06:54:10.297: INFO: Container ebs-plugin ready: true, restart count 0 Jan 24 06:54:10.297: INFO: Container liveness-probe ready: true, restart count 0 Jan 24 06:54:10.297: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 24 06:54:10.297: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-fgc56 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:10.297: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2v7c5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:10.297: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-4sk4h started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:10.297: INFO: hostexec-i-0697a70b9aa95b7c0-n9ccb started at 2023-01-24 06:49:56 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container agnhost-container ready: true, restart count 0 Jan 24 06:54:10.297: INFO: service-headless-xh4pl started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container service-headless ready: false, restart count 0 Jan 24 06:54:10.297: INFO: csi-mockplugin-0 started at 2023-01-24 06:48:54 +0000 UTC (0+4 container statuses recorded) Jan 24 06:54:10.297: INFO: Container busybox ready: false, restart count 0 Jan 24 06:54:10.297: INFO: Container csi-provisioner ready: false, restart count 0 Jan 24 06:54:10.297: INFO: Container driver-registrar ready: false, restart count 0 Jan 24 06:54:10.297: INFO: Container mock ready: false, restart count 0 Jan 24 06:54:10.297: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-68w92 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:10.297: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qphds started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:10.297: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qtzq5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:10.297: INFO: rs-b9vhw started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container donothing ready: true, restart count 0 Jan 24 06:54:10.297: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-2tkvs started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0 Jan 24 06:54:10.297: INFO: cilium-dh7hm started at 2023-01-24 06:27:36 +0000 UTC (1+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 24 06:54:10.297: INFO: Container cilium-agent ready: true, restart count 0 Jan 24 06:54:10.297: INFO: coredns-559769c974-6wblg started at 2023-01-24 06:27:49 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container coredns ready: true, restart count 0 Jan 24 06:54:10.297: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-5pcw5 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:10.297: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cdkdr started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:10.297: INFO: rs-w9wjl started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container donothing ready: true, restart count 0 Jan 24 06:54:10.297: INFO: pod-subpath-test-preprovisionedpv-94gx started at 2023-01-24 06:50:18 +0000 UTC (2+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Init container init-volume-preprovisionedpv-94gx ready: false, restart count 0 Jan 24 06:54:10.297: INFO: Init container test-init-volume-preprovisionedpv-94gx ready: false, restart count 0 Jan 24 06:54:10.297: INFO: Container test-container-subpath-preprovisionedpv-94gx ready: false, restart count 0 Jan 24 06:54:10.297: INFO: coredns-autoscaler-7cb5c5b969-vktlf started at 2023-01-24 06:27:49 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container autoscaler ready: true, restart count 0 Jan 24 06:54:10.297: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-xbpd8 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:10.297: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0 Jan 24 06:54:10.922: INFO: Latency metrics for node i-0697a70b9aa95b7c0 Jan 24 06:54:10.922: INFO: Logging node info for node i-0b0b3ac061164cea4 Jan 24 06:54:11.077: INFO: Node Info: &Node{ObjectMeta:{i-0b0b3ac061164cea4 ac00aaf7-4427-49f6-b849-9c812ae80069 34896 0 2023-01-24 06:25:23 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:c5.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0b0b3ac061164cea4 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c5.large topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.51.48 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0b0b3ac061164cea4"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-24 06:25:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-24 06:25:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:26:22 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:26:22 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-24 06:26:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-24 06:53:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0b0b3ac061164cea4,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3892297728 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3787440128 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:53:16 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:53:16 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:53:16 +0000 UTC,LastTransitionTime:2023-01-24 06:25:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:53:16 +0000 UTC,LastTransitionTime:2023-01-24 06:26:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.51.48,},NodeAddress{Type:ExternalIP,Address:13.125.222.159,},NodeAddress{Type:InternalDNS,Address:i-0b0b3ac061164cea4.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0b0b3ac061164cea4.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-13-125-222-159.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec255807a7dec89d2cc9cfee6d481c20,SystemUUID:ec255807-a7de-c89d-2cc9-cfee6d481c20,BootID:4153aea0-5d07-4f37-a966-c56ef660acfc,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.26.1],SizeBytes:135178704,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.26.1],SizeBytes:124995897,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:102157811,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:100947667,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:99868722,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:98888614,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:86742272,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:85524408,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:76162358,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:63920476,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:63423287,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.26.1],SizeBytes:57661752,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:44339030,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:42841305,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:26802430,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:23345856,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:22381475,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:22085298,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:20154862,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4965794,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 24 06:54:11.077: INFO: Logging kubelet events for node i-0b0b3ac061164cea4 Jan 24 06:54:11.235: INFO: Logging pods the kubelet thinks is on node i-0b0b3ac061164cea4 Jan 24 06:54:11.396: INFO: dns-controller-69987775c6-vzqc8 started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:11.396: INFO: Container dns-controller ready: true, restart count 0 Jan 24 06:54:11.396: INFO: ebs-csi-controller-5bd98b456f-kswjq started at 2023-01-24 06:26:07 +0000 UTC (0+5 container statuses recorded) Jan 24 06:54:11.396: INFO: Container csi-attacher ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Container csi-provisioner ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Container csi-resizer ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Container ebs-plugin ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Container liveness-probe ready: true, restart count 0 Jan 24 06:54:11.396: INFO: kube-apiserver-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+2 container statuses recorded) Jan 24 06:54:11.396: INFO: Container healthcheck ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Container kube-apiserver ready: true, restart count 3 Jan 24 06:54:11.396: INFO: aws-cloud-controller-manager-4n9pv started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:11.396: INFO: Container aws-cloud-controller-manager ready: true, restart count 0 Jan 24 06:54:11.396: INFO: ebs-csi-node-q4jsj started at 2023-01-24 06:26:07 +0000 UTC (0+3 container statuses recorded) Jan 24 06:54:11.396: INFO: Container ebs-plugin ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Container liveness-probe ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 24 06:54:11.396: INFO: kops-controller-vfj77 started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:11.396: INFO: Container kops-controller ready: true, restart count 0 Jan 24 06:54:11.396: INFO: cilium-w225f started at 2023-01-24 06:26:07 +0000 UTC (1+1 container statuses recorded) Jan 24 06:54:11.396: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Container cilium-agent ready: true, restart count 0 Jan 24 06:54:11.396: INFO: cilium-operator-5dd44dc49f-rn9bl started at 2023-01-24 06:26:07 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:11.396: INFO: Container cilium-operator ready: true, restart count 0 Jan 24 06:54:11.396: INFO: etcd-manager-events-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (10+1 container statuses recorded) Jan 24 06:54:11.396: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Container etcd-manager ready: true, restart count 0 Jan 24 06:54:11.396: INFO: etcd-manager-main-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (10+1 container statuses recorded) Jan 24 06:54:11.396: INFO: Init container init-etcd-3-2-24 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-3-10 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-3-17 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-4-13 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-4-3 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-5-0 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-5-1 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-5-3 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-5-4 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Init container init-etcd-3-5-6 ready: true, restart count 0 Jan 24 06:54:11.396: INFO: Container etcd-manager ready: true, restart count 0 Jan 24 06:54:11.396: INFO: kube-controller-manager-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:11.396: INFO: Container kube-controller-manager ready: true, restart count 4 Jan 24 06:54:11.396: INFO: kube-scheduler-i-0b0b3ac061164cea4 started at 2023-01-24 06:23:42 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:11.396: INFO: Container kube-scheduler ready: true, restart count 0 Jan 24 06:54:11.925: INFO: Latency metrics for node i-0b0b3ac061164cea4 Jan 24 06:54:11.925: INFO: Logging node info for node i-0e0a289a9d5c333a0 Jan 24 06:54:12.080: INFO: Node Info: &Node{ObjectMeta:{i-0e0a289a9d5c333a0 9194794b-d78a-4a30-bbec-b76df63f68f1 33830 0 2023-01-24 06:27:40 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0e0a289a9d5c333a0 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0e0a289a9d5c333a0 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.61.62 csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1968":"i-0e0a289a9d5c333a0","ebs.csi.aws.com":"i-0e0a289a9d5c333a0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:49:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0e0a289a9d5c333a0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:40 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:49:26 +0000 UTC,LastTransitionTime:2023-01-24 06:27:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.61.62,},NodeAddress{Type:ExternalIP,Address:54.180.158.26,},NodeAddress{Type:InternalDNS,Address:i-0e0a289a9d5c333a0.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0e0a289a9d5c333a0.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-54-180-158-26.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2281a7b00c77d27e8055a22bf9facf,SystemUUID:ec2281a7-b00c-77d2-7e80-55a22bf9facf,BootID:be2e68ba-6839-4b53-847f-22f29f0a7360,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1968^3d6e646f-9bb3-11ed-bc6b-56b051ebca30],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 24 06:54:12.080: INFO: Logging kubelet events for node i-0e0a289a9d5c333a0 Jan 24 06:54:12.238: INFO: Logging pods the kubelet thinks is on node i-0e0a289a9d5c333a0 Jan 24 06:54:12.399: INFO: test-ss-0 started at 2023-01-24 06:49:10 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container webserver ready: false, restart count 0 Jan 24 06:54:12.400: INFO: dns-test-b3d6c428-d0cf-4d5a-9700-fdfb874b5d91 started at 2023-01-24 06:48:41 +0000 UTC (0+3 container statuses recorded) Jan 24 06:54:12.400: INFO: Container jessie-querier ready: false, restart count 0 Jan 24 06:54:12.400: INFO: Container querier ready: false, restart count 0 Jan 24 06:54:12.400: INFO: Container webserver ready: false, restart count 0 Jan 24 06:54:12.400: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-9hr5l started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:12.400: INFO: e2e-test-httpd-pod started at 2023-01-24 06:49:16 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container e2e-test-httpd-pod ready: false, restart count 0 Jan 24 06:54:12.400: INFO: pod1 started at 2023-01-24 06:50:50 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container agnhost-container ready: false, restart count 0 Jan 24 06:54:12.400: INFO: service-headless-h4z88 started at 2023-01-24 06:44:49 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container service-headless ready: true, restart count 0 Jan 24 06:54:12.400: INFO: rs-l5fgd started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container donothing ready: false, restart count 0 Jan 24 06:54:12.400: INFO: cilium-cvt86 started at 2023-01-24 06:27:41 +0000 UTC (1+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 24 06:54:12.400: INFO: Container cilium-agent ready: true, restart count 0 Jan 24 06:54:12.400: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-224qm started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:12.400: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-z9lc4 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:12.400: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-7nhf8 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:12.400: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-w7m2k started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: false, restart count 0 Jan 24 06:54:12.400: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-867tg started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:12.400: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-zv4q6 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:12.400: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-tbbms started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:12.400: INFO: pvc-volume-tester-56wjf started at 2023-01-24 06:49:25 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container volume-tester ready: false, restart count 0 Jan 24 06:54:12.400: INFO: ebs-csi-node-v6prc started at 2023-01-24 06:27:41 +0000 UTC (0+3 container statuses recorded) Jan 24 06:54:12.400: INFO: Container ebs-plugin ready: true, restart count 0 Jan 24 06:54:12.400: INFO: Container liveness-probe ready: true, restart count 0 Jan 24 06:54:12.400: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 24 06:54:12.400: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-ts8xr started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:12.400: INFO: rs-rsqdt started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container donothing ready: true, restart count 0 Jan 24 06:54:12.400: INFO: tester started at 2023-01-24 06:53:16 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container tester ready: false, restart count 0 Jan 24 06:54:12.400: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-bkjzk started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:12.400: INFO: csi-mockplugin-0 started at 2023-01-24 06:48:18 +0000 UTC (0+3 container statuses recorded) Jan 24 06:54:12.400: INFO: Container csi-provisioner ready: true, restart count 0 Jan 24 06:54:12.400: INFO: Container driver-registrar ready: true, restart count 0 Jan 24 06:54:12.400: INFO: Container mock ready: true, restart count 0 Jan 24 06:54:12.400: INFO: hostexec-i-0e0a289a9d5c333a0-7tx4h started at 2023-01-24 06:53:15 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container agnhost-container ready: true, restart count 0 Jan 24 06:54:12.400: INFO: rs-znpr8 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container donothing ready: false, restart count 0 Jan 24 06:54:12.400: INFO: pod-1 started at 2023-01-24 06:53:19 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container donothing ready: false, restart count 0 Jan 24 06:54:12.400: INFO: ss-0 started at 2023-01-24 06:53:26 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container webserver ready: false, restart count 0 Jan 24 06:54:12.400: INFO: pod-subpath-test-preprovisionedpv-qftm started at 2023-01-24 06:53:33 +0000 UTC (1+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Init container init-volume-preprovisionedpv-qftm ready: false, restart count 0 Jan 24 06:54:12.400: INFO: Container test-container-subpath-preprovisionedpv-qftm ready: false, restart count 0 Jan 24 06:54:12.400: INFO: csi-mockplugin-attacher-0 started at 2023-01-24 06:48:19 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:12.400: INFO: Container csi-attacher ready: false, restart count 0 Jan 24 06:54:13.004: INFO: Latency metrics for node i-0e0a289a9d5c333a0 Jan 24 06:54:13.004: INFO: Logging node info for node i-0f5ee4c33b6799e76 Jan 24 06:54:13.159: INFO: Node Info: &Node{ObjectMeta:{i-0f5ee4c33b6799e76 34c5957d-82ce-41f2-824d-90a78fa8bbba 33946 0 2023-01-24 06:27:39 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:t3.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-northeast-2 failure-domain.beta.kubernetes.io/zone:ap-northeast-2a io.kubernetes.storage.mock/node:some-mock-node kubelet_cleanup:true kubernetes.io/arch:amd64 kubernetes.io/hostname:i-0f5ee4c33b6799e76 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t3.medium topology.ebs.csi.aws.com/zone:ap-northeast-2a topology.hostpath.csi/node:i-0f5ee4c33b6799e76 topology.kubernetes.io/region:ap-northeast-2 topology.kubernetes.io/zone:ap-northeast-2a] map[alpha.kubernetes.io/provided-node-ip:172.20.57.132 csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0f5ee4c33b6799e76"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-24 06:27:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:alpha.kubernetes.io/provided-node-ip":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {e2e.test Update v1 2023-01-24 06:42:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubelet_cleanup":{}}}} } {kubelet Update v1 2023-01-24 06:49:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ap-northeast-2a/i-0f5ee4c33b6799e76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49753808896 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{4051677184 0} {<nil>} 3956716Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44778427933 0} {<nil>} 44778427933 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3946819584 0} {<nil>} 3854316Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:39 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-24 06:49:25 +0000 UTC,LastTransitionTime:2023-01-24 06:27:55 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.57.132,},NodeAddress{Type:ExternalIP,Address:3.36.124.131,},NodeAddress{Type:InternalDNS,Address:i-0f5ee4c33b6799e76.ap-northeast-2.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f5ee4c33b6799e76.ap-northeast-2.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-36-124-131.ap-northeast-2.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2f5bcf8e071d2b94271b14b41c978c,SystemUUID:ec2f5bcf-8e07-1d2b-9427-1b14b41c978c,BootID:400ceb00-7274-4349-89f8-a3cb13aed9c2,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 20.04.5 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.26.1,KubeProxyVersion:v1.26.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:166719855,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.26.1],SizeBytes:67205316,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:29725845,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:15273057,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:8240918,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:301773,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Jan 24 06:54:13.160: INFO: Logging kubelet events for node i-0f5ee4c33b6799e76 Jan 24 06:54:13.317: INFO: Logging pods the kubelet thinks is on node i-0f5ee4c33b6799e76 Jan 24 06:54:13.478: INFO: pod-subpath-test-inlinevolume-src5 started at 2023-01-24 06:53:52 +0000 UTC (1+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Init container init-volume-inlinevolume-src5 ready: false, restart count 0 Jan 24 06:54:13.478: INFO: Container test-container-subpath-inlinevolume-src5 ready: false, restart count 0 Jan 24 06:54:13.478: INFO: pod-failure-failjob-s2mc2 started at 2023-01-24 06:49:00 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Container c ready: false, restart count 0 Jan 24 06:54:13.478: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-qmz8q started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:13.478: INFO: cilium-ql489 started at 2023-01-24 06:27:40 +0000 UTC (1+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Init container clean-cilium-state ready: true, restart count 0 Jan 24 06:54:13.478: INFO: Container cilium-agent ready: true, restart count 0 Jan 24 06:54:13.478: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-hwktw started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:13.478: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-cfnfh started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:13.478: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-xl6bg started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:13.478: INFO: rs-g6rhr started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Container donothing ready: true, restart count 0 Jan 24 06:54:13.478: INFO: rs-8b8n9 started at 2023-01-24 06:44:14 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Container donothing ready: false, restart count 0 Jan 24 06:54:13.478: INFO: coredns-559769c974-5gltc started at 2023-01-24 06:28:16 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Container coredns ready: true, restart count 0 Jan 24 06:54:13.478: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-7lw99 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:13.478: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-w9729 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:13.478: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-lvvw2 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:13.478: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-wjt45 started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:13.478: INFO: ebs-csi-node-vjhdb started at 2023-01-24 06:27:40 +0000 UTC (0+3 container statuses recorded) Jan 24 06:54:13.478: INFO: Container ebs-plugin ready: true, restart count 0 Jan 24 06:54:13.478: INFO: Container liveness-probe ready: true, restart count 0 Jan 24 06:54:13.478: INFO: Container node-driver-registrar ready: true, restart count 0 Jan 24 06:54:13.478: INFO: pod-failure-failjob-s22zv started at 2023-01-24 06:49:00 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Container c ready: false, restart count 0 Jan 24 06:54:13.478: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-r5vgx started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:13.478: INFO: cleanup40-351538ae-756b-458d-abb0-ac872b41ab58-6lkqq started at 2023-01-24 06:42:29 +0000 UTC (0+1 container statuses recorded) Jan 24 06:54:13.478: INFO: Container cleanup40-351538ae-756b-458d-abb0-ac872b41ab58 ready: true, restart count 0 Jan 24 06:54:14.040: INFO: Latency metrics for node i-0f5ee4c33b6799e76 [DeferCleanup (Each)] [sig-network] Services tear down framework | framework.go:193 �[1mSTEP:�[0m Destroying namespace "services-6896" for this suite. �[38;5;243m01/24/23 06:54:14.04�[0m
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-node\]\sContainer\sLifecycle\sHook\swhen\screate\sa\spod\swith\slifecycle\shook\sshould\sexecute\sprestop\sexec\shook\sproperly\s\[NodeConformance\]\s\[Conformance\]$'
test/e2e/framework/pod/pod_client.go:107 k8s.io/kubernetes/test/e2e/framework/pod.(*PodClient).CreateSync(0xc000ccf908, 0x38?) test/e2e/framework/pod/pod_client.go:107 +0x94 k8s.io/kubernetes/test/e2e/common/node.glob..func12.1.1() test/e2e/common/node/lifecycle_hook.go:87 +0x3da
[BeforeEach] [sig-node] Container Lifecycle Hook set up framework | framework.go:178 �[1mSTEP:�[0m Creating a kubernetes client �[38;5;243m01/24/23 06:48:14.44�[0m Jan 24 06:48:14.440: INFO: >>> kubeConfig: /root/.kube/config �[1mSTEP:�[0m Building a namespace api object, basename container-lifecycle-hook �[38;5;243m01/24/23 06:48:14.441�[0m �[1mSTEP:�[0m Waiting for a default service account to be provisioned in namespace �[38;5;243m01/24/23 06:48:14.916�[0m �[1mSTEP:�[0m Waiting for kube-root-ca.crt to be provisioned in namespace �[38;5;243m01/24/23 06:48:15.232�[0m [BeforeEach] [sig-node] Container Lifecycle Hook test/e2e/framework/metrics/init/init.go:31 [BeforeEach] when create a pod with lifecycle hook test/e2e/common/node/lifecycle_hook.go:77 �[1mSTEP:�[0m create the container to handle the HTTPGet hook request. �[38;5;243m01/24/23 06:48:15.707�[0m Jan 24 06:48:15.868: INFO: Waiting up to 5m0s for pod "pod-handle-http-request" in namespace "container-lifecycle-hook-1558" to be "running and ready" Jan 24 06:48:16.026: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 158.316916ms Jan 24 06:48:16.026: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:18.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2.316781564s Jan 24 06:48:18.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:20.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4.317954863s Jan 24 06:48:20.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:22.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 6.31699388s Jan 24 06:48:22.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:24.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 8.318243857s Jan 24 06:48:24.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:26.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 10.318330457s Jan 24 06:48:26.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:28.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 12.318182026s Jan 24 06:48:28.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:30.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 14.31718071s Jan 24 06:48:30.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:32.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 16.317266844s Jan 24 06:48:32.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:34.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 18.318274898s Jan 24 06:48:34.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:36.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 20.31691173s Jan 24 06:48:36.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:38.187: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 22.318728558s Jan 24 06:48:38.187: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:40.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 24.317477001s Jan 24 06:48:40.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:42.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 26.317668581s Jan 24 06:48:42.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:44.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 28.318241911s Jan 24 06:48:44.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:46.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 30.318021986s Jan 24 06:48:46.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:48.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 32.31811299s Jan 24 06:48:48.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:50.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 34.317920089s Jan 24 06:48:50.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:52.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 36.317633643s Jan 24 06:48:52.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:54.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 38.31820918s Jan 24 06:48:54.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:56.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 40.317708967s Jan 24 06:48:56.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:48:58.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 42.317381467s Jan 24 06:48:58.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:00.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 44.317720787s Jan 24 06:49:00.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:02.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 46.317358515s Jan 24 06:49:02.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:04.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 48.31750415s Jan 24 06:49:04.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:06.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 50.316967719s Jan 24 06:49:06.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:08.193: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 52.324492294s Jan 24 06:49:08.193: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:10.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 54.317668513s Jan 24 06:49:10.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:12.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 56.318002145s Jan 24 06:49:12.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:14.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 58.318093024s Jan 24 06:49:14.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:16.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.316995816s Jan 24 06:49:16.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:18.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.317854042s Jan 24 06:49:18.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:20.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.317733s Jan 24 06:49:20.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:22.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.31695935s Jan 24 06:49:22.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:24.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.317477356s Jan 24 06:49:24.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:26.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.317920241s Jan 24 06:49:26.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:28.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.317890743s Jan 24 06:49:28.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:30.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.317627688s Jan 24 06:49:30.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:32.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.317916226s Jan 24 06:49:32.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:34.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.317009729s Jan 24 06:49:34.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:36.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.317826702s Jan 24 06:49:36.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:38.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.317620908s Jan 24 06:49:38.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:40.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.317044275s Jan 24 06:49:40.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:42.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.318306245s Jan 24 06:49:42.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:44.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.317979586s Jan 24 06:49:44.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:46.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.318104262s Jan 24 06:49:46.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:48.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.317605581s Jan 24 06:49:48.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:50.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.317835131s Jan 24 06:49:50.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:52.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.317447816s Jan 24 06:49:52.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:54.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.318061917s Jan 24 06:49:54.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:56.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.316863671s Jan 24 06:49:56.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:49:58.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.317629063s Jan 24 06:49:58.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:00.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.31709276s Jan 24 06:50:00.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:02.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.318151274s Jan 24 06:50:02.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:04.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.317382728s Jan 24 06:50:04.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:06.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.318213367s Jan 24 06:50:06.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:08.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.31721199s Jan 24 06:50:08.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:10.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.31709274s Jan 24 06:50:10.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:12.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.316752105s Jan 24 06:50:12.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:14.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.31740245s Jan 24 06:50:14.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:16.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.318256839s Jan 24 06:50:16.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:18.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.317200823s Jan 24 06:50:18.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:20.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.31829122s Jan 24 06:50:20.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:22.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.318118299s Jan 24 06:50:22.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:24.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.317653759s Jan 24 06:50:24.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:26.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.317954845s Jan 24 06:50:26.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:28.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.317045077s Jan 24 06:50:28.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:30.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.317673212s Jan 24 06:50:30.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:32.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.317246473s Jan 24 06:50:32.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:34.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.317185446s Jan 24 06:50:34.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:36.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.318277271s Jan 24 06:50:36.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:38.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.317317746s Jan 24 06:50:38.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:40.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.316861147s Jan 24 06:50:40.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:42.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.318079607s Jan 24 06:50:42.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:44.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.318149345s Jan 24 06:50:44.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:46.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.318203235s Jan 24 06:50:46.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:48.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.317971076s Jan 24 06:50:48.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:50.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.316855254s Jan 24 06:50:50.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:52.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.318045894s Jan 24 06:50:52.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:54.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.317947278s Jan 24 06:50:54.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:56.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.317922935s Jan 24 06:50:56.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:50:58.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.317030615s Jan 24 06:50:58.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:00.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.317492426s Jan 24 06:51:00.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:02.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.31719788s Jan 24 06:51:02.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:04.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.317980051s Jan 24 06:51:04.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:06.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.317905813s Jan 24 06:51:06.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:08.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.318239155s Jan 24 06:51:08.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:10.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.317146571s Jan 24 06:51:10.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:12.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.316899967s Jan 24 06:51:12.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:14.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.318092172s Jan 24 06:51:14.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:16.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.316803716s Jan 24 06:51:16.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:18.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.317529315s Jan 24 06:51:18.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:20.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.318038119s Jan 24 06:51:20.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:22.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.317874835s Jan 24 06:51:22.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:24.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.317011335s Jan 24 06:51:24.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:26.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.317466313s Jan 24 06:51:26.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:28.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.317194689s Jan 24 06:51:28.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:30.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.317973838s Jan 24 06:51:30.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:32.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.318064224s Jan 24 06:51:32.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:34.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.317864249s Jan 24 06:51:34.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:36.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.318039717s Jan 24 06:51:36.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:38.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.317456869s Jan 24 06:51:38.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:40.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.317595886s Jan 24 06:51:40.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:42.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.317048594s Jan 24 06:51:42.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:44.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.31713803s Jan 24 06:51:44.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:46.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.317025447s Jan 24 06:51:46.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:48.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.317471615s Jan 24 06:51:48.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:50.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.316939121s Jan 24 06:51:50.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:52.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.316998439s Jan 24 06:51:52.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:54.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.317388939s Jan 24 06:51:54.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:56.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.31809331s Jan 24 06:51:56.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:51:58.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.317804184s Jan 24 06:51:58.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:00.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.317256614s Jan 24 06:52:00.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:02.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.318059679s Jan 24 06:52:02.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:04.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.318093552s Jan 24 06:52:04.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:06.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.317128115s Jan 24 06:52:06.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:08.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.317722787s Jan 24 06:52:08.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:10.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.317709433s Jan 24 06:52:10.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:12.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.317140437s Jan 24 06:52:12.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:14.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.317200299s Jan 24 06:52:14.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:16.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.317454959s Jan 24 06:52:16.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:18.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.317608297s Jan 24 06:52:18.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:20.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.318013031s Jan 24 06:52:20.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:22.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.318095338s Jan 24 06:52:22.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:24.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.318065607s Jan 24 06:52:24.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:26.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.317853439s Jan 24 06:52:26.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:28.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.317045135s Jan 24 06:52:28.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:30.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.317138877s Jan 24 06:52:30.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:32.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.318137569s Jan 24 06:52:32.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:34.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.317727978s Jan 24 06:52:34.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:36.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.317764851s Jan 24 06:52:36.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:38.187: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.318371971s Jan 24 06:52:38.187: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:40.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.317989902s Jan 24 06:52:40.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:42.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.317040298s Jan 24 06:52:42.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:44.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.3180329s Jan 24 06:52:44.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:46.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.317007888s Jan 24 06:52:46.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:48.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.317422332s Jan 24 06:52:48.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:50.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.316780307s Jan 24 06:52:50.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:52.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.318056303s Jan 24 06:52:52.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:54.188: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.319560618s Jan 24 06:52:54.188: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:56.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.317290595s Jan 24 06:52:56.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:52:58.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.317303709s Jan 24 06:52:58.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:53:00.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.317089154s Jan 24 06:53:00.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:53:02.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.318182798s Jan 24 06:53:02.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:53:04.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.318049142s Jan 24 06:53:04.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:53:06.185: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.31692369s Jan 24 06:53:06.185: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:53:08.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.317779575s Jan 24 06:53:08.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:53:10.186: INFO: Pod "pod-handle-http-request": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.317984893s Jan 24 06:53:10.186: INFO: The phase of Pod pod-handle-http-request is Pending, waiting for it to be Running (with Ready = true) Jan 24 06:53:12.185: INFO: Pod "pod-handle-http-request": Phase=&