e2e-kops-warm-pool #1619227358076604416 Results

Test Failures

Kubernetes e2e suite [It] [sig-node] RuntimeClass should run a Pod requesting a RuntimeClass with a configured handler [NodeFeature:RuntimeHandler] 5m4s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-node\]\sRuntimeClass\sshould\srun\sa\sPod\srequesting\sa\sRuntimeClass\swith\sa\sconfigured\shandler\s\[NodeFeature\:RuntimeHandler\]$'

[FAILED] timed out while waiting for pod runtimeclass-1166/test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c to be Succeeded or Failed
In [It] at: test/e2e/common/node/runtimeclass.go:386 @ 01/28/23 07:24:18.143

				> Enter [BeforeEach] [sig-node] RuntimeClass - set up framework | framework.go:188 @ 01/28/23 07:19:15.317
STEP: Creating a kubernetes client - test/e2e/framework/framework.go:208 @ 01/28/23 07:19:15.317
Jan 28 07:19:15.317: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename runtimeclass - test/e2e/framework/framework.go:247 @ 01/28/23 07:19:15.319
STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:256 @ 01/28/23 07:19:15.424
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 01/28/23 07:19:15.486
< Exit [BeforeEach] [sig-node] RuntimeClass - set up framework | framework.go:188 @ 01/28/23 07:19:15.551 (234ms)
> Enter [BeforeEach] [sig-node] RuntimeClass - test/e2e/framework/metrics/init/init.go:33 @ 01/28/23 07:19:15.551
< Exit [BeforeEach] [sig-node] RuntimeClass - test/e2e/framework/metrics/init/init.go:33 @ 01/28/23 07:19:15.552 (0s)
> Enter [It] should run a Pod requesting a RuntimeClass with a configured handler [NodeFeature:RuntimeHandler] - test/e2e/common/node/runtimeclass.go:85 @ 01/28/23 07:19:15.552
Jan 28 07:19:15.619: INFO: Waiting up to 5m0s for pod "hostexec-i-0ab74bbb764104081-dtt48" in namespace "runtimeclass-1166" to be "running"
Jan 28 07:19:15.650: INFO: Pod "hostexec-i-0ab74bbb764104081-dtt48": Phase="Pending", Reason="", readiness=false. Elapsed: 31.229853ms
Jan 28 07:19:17.683: INFO: Pod "hostexec-i-0ab74bbb764104081-dtt48": Phase="Running", Reason="", readiness=true. Elapsed: 2.064645406s
Jan 28 07:19:17.683: INFO: Pod "hostexec-i-0ab74bbb764104081-dtt48" satisfied condition "running"
Jan 28 07:19:17.683: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c if [ -e '/etc/containerd/config.toml' ]; then
grep -q 'runtimes.test-handler' /etc/containerd/config.toml
  return
fi

if [ -e '/etc/crio/crio.conf' ]; then
  grep -q 'runtimes.test-handler' /etc/crio/crio.conf
  return
fi
] Namespace:runtimeclass-1166 PodName:hostexec-i-0ab74bbb764104081-dtt48 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true Quiet:false}
Jan 28 07:19:17.683: INFO: >>> kubeConfig: /root/.kube/config
Jan 28 07:19:17.684: INFO: ExecWithOptions: Clientset creation
Jan 28 07:19:17.684: INFO: ExecWithOptions: execute(POST https://api-e2e-e2e-kops-warm-poo-agujp3-f36db33deca21475.elb.ca-central-1.amazonaws.com/api/v1/namespaces/runtimeclass-1166/pods/hostexec-i-0ab74bbb764104081-dtt48/exec?command=nsenter&command=--mount%3D%2Frootfs%2Fproc%2F1%2Fns%2Fmnt&command=--&command=sh&command=-c&command=if+%5B+-e+%27%2Fetc%2Fcontainerd%2Fconfig.toml%27+%5D%3B+then%0Agrep+-q+%27runtimes.test-handler%27+%2Fetc%2Fcontainerd%2Fconfig.toml%0A++return%0Afi%0A%0Aif+%5B+-e+%27%2Fetc%2Fcrio%2Fcrio.conf%27+%5D%3B+then%0A++grep+-q+%27runtimes.test-handler%27+%2Fetc%2Fcrio%2Fcrio.conf%0A++return%0Afi%0A&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true)
Jan 28 07:19:18.046: INFO: Waiting up to 5m0s for pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c" in namespace "runtimeclass-1166" to be "Succeeded or Failed"
Jan 28 07:19:18.078: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 31.497642ms
Jan 28 07:19:20.125: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2.078974005s
Jan 28 07:19:22.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4.06366071s
Jan 28 07:19:24.114: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 6.067682547s
Jan 28 07:19:26.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 8.06302125s
Jan 28 07:19:28.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 10.063425072s
Jan 28 07:19:30.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 12.06332504s
Jan 28 07:19:32.111: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 14.06426989s
Jan 28 07:19:34.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 16.063943154s
Jan 28 07:19:36.111: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 18.065017434s
Jan 28 07:19:38.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 20.064026067s
Jan 28 07:19:40.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 22.063462318s
Jan 28 07:19:42.111: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 24.064880137s
Jan 28 07:19:44.111: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 26.064388653s
Jan 28 07:19:46.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 28.063146435s
Jan 28 07:19:48.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 30.063222477s
Jan 28 07:19:50.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 32.063668704s
Jan 28 07:19:52.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 34.063478041s
Jan 28 07:19:54.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 36.064112847s
Jan 28 07:19:56.111: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 38.064422249s
Jan 28 07:19:58.111: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 40.064517384s
Jan 28 07:20:00.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 42.063267779s
Jan 28 07:20:02.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 44.064236572s
Jan 28 07:20:04.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 46.063586633s
Jan 28 07:20:06.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 48.063722124s
Jan 28 07:20:08.111: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 50.064412113s
Jan 28 07:20:10.115: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 52.068541704s
Jan 28 07:20:12.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 54.063449832s
Jan 28 07:20:14.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 56.063999218s
Jan 28 07:20:16.118: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 58.072267874s
Jan 28 07:20:18.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.063474832s
Jan 28 07:20:20.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.063408243s
Jan 28 07:20:22.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.064264696s
Jan 28 07:20:24.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.063318027s
Jan 28 07:20:26.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.064040032s
Jan 28 07:20:28.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.063360811s
Jan 28 07:20:30.119: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.072503921s
Jan 28 07:20:32.117: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.071016162s
Jan 28 07:20:34.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.063285282s
Jan 28 07:20:36.113: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.066898641s
Jan 28 07:20:38.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.063820837s
Jan 28 07:20:40.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.063354406s
Jan 28 07:20:42.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.063173707s
Jan 28 07:20:44.111: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.064709422s
Jan 28 07:20:46.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.063684491s
Jan 28 07:20:48.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.063619497s
Jan 28 07:20:50.111: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.064404594s
Jan 28 07:20:52.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.064096839s
Jan 28 07:20:54.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.063232231s
Jan 28 07:20:56.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.063125335s
Jan 28 07:20:58.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.063596422s
Jan 28 07:21:00.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.06351302s
Jan 28 07:21:02.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.063463325s
Jan 28 07:21:04.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.063246147s
Jan 28 07:21:06.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.063716992s
Jan 28 07:21:08.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.064053267s
Jan 28 07:21:10.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.063662313s
Jan 28 07:21:12.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.062913576s
Jan 28 07:21:14.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.063634562s
Jan 28 07:21:16.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.063777454s
Jan 28 07:21:18.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.064094433s
Jan 28 07:21:20.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.063656714s
Jan 28 07:21:22.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.063342052s
Jan 28 07:21:24.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.063479435s
Jan 28 07:21:26.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.063242773s
Jan 28 07:21:28.111: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.065047228s
Jan 28 07:21:30.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.063548612s
Jan 28 07:21:32.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.063816241s
Jan 28 07:21:34.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.064111547s
Jan 28 07:21:36.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.063957776s
Jan 28 07:21:38.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.063550835s
Jan 28 07:21:40.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.063397362s
Jan 28 07:21:42.111: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.064638532s
Jan 28 07:21:44.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.063362869s
Jan 28 07:21:46.119: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.072556347s
Jan 28 07:21:48.113: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.067210417s
Jan 28 07:21:50.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.063269933s
Jan 28 07:21:52.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.063241354s
Jan 28 07:21:54.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.063319899s
Jan 28 07:21:56.111: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.06518165s
Jan 28 07:21:58.125: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.078434368s
Jan 28 07:22:00.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.063814183s
Jan 28 07:22:02.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.064092442s
Jan 28 07:22:04.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.063800629s
Jan 28 07:22:06.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.063828388s
Jan 28 07:22:08.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.063672346s
Jan 28 07:22:10.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.063534415s
Jan 28 07:22:12.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.064204473s
Jan 28 07:22:14.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.063850275s
Jan 28 07:22:16.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.063943151s
Jan 28 07:22:18.111: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.064269547s
Jan 28 07:22:20.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.063512628s
Jan 28 07:22:22.111: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.064402776s
Jan 28 07:22:24.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.063613514s
Jan 28 07:22:26.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.063314384s
Jan 28 07:22:28.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.063478129s
Jan 28 07:22:30.112: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.065634902s
Jan 28 07:22:32.113: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.066776698s
Jan 28 07:22:34.113: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.066373066s
Jan 28 07:22:36.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.063630493s
Jan 28 07:22:38.113: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.066871395s
Jan 28 07:22:40.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.063210899s
Jan 28 07:22:42.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.063357416s
Jan 28 07:22:44.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.063607374s
Jan 28 07:22:46.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.063131848s
Jan 28 07:22:48.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.063768822s
Jan 28 07:22:50.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.064052142s
Jan 28 07:22:52.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.064232352s
Jan 28 07:22:54.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.0637353s
Jan 28 07:22:56.112: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.065898853s
Jan 28 07:22:58.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.063186448s
Jan 28 07:23:00.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.063785817s
Jan 28 07:23:02.115: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.068400668s
Jan 28 07:23:04.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.063682718s
Jan 28 07:23:06.111: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.064548598s
Jan 28 07:23:08.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.063842173s
Jan 28 07:23:10.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.063990534s
Jan 28 07:23:12.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.063751313s
Jan 28 07:23:14.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.063675475s
Jan 28 07:23:16.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.063750179s
Jan 28 07:23:18.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.063291677s
Jan 28 07:23:20.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.064142316s
Jan 28 07:23:22.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.063398669s
Jan 28 07:23:24.113: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.06703346s
Jan 28 07:23:26.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.063592206s
Jan 28 07:23:28.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.063415477s
Jan 28 07:23:30.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.063302768s
Jan 28 07:23:32.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.06313288s
Jan 28 07:23:34.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.063389657s
Jan 28 07:23:36.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.064117299s
Jan 28 07:23:38.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.063650318s
Jan 28 07:23:40.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.063674661s
Jan 28 07:23:42.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.063472833s
Jan 28 07:23:44.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.063729641s
Jan 28 07:23:46.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.063875844s
Jan 28 07:23:48.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.063737233s
Jan 28 07:23:50.113: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.066295495s
Jan 28 07:23:52.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.063618449s
Jan 28 07:23:54.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.063789668s
Jan 28 07:23:56.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.064066441s
Jan 28 07:23:58.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.064191314s
Jan 28 07:24:00.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.063217187s
Jan 28 07:24:02.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.063545093s
Jan 28 07:24:04.109: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.063091758s
Jan 28 07:24:06.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.063705018s
Jan 28 07:24:08.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.063620685s
Jan 28 07:24:10.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.063836405s
Jan 28 07:24:12.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.063578468s
Jan 28 07:24:14.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.063396223s
Jan 28 07:24:16.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.064256221s
Jan 28 07:24:18.110: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.063899184s
Jan 28 07:24:18.142: INFO: Pod "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.095506091s
Jan 28 07:24:18.143: INFO: Unexpected error: 
    <*pod.timeoutError | 0xc003e4f200>: {
        msg: "timed out while waiting for pod runtimeclass-1166/test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c to be Succeeded or Failed",
        observedObjects: [
            <*v1.Pod | 0xc00068c480>{
                TypeMeta: {Kind: "", APIVersion: ""},
                ObjectMeta: {
                    Name: "test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c",
                    GenerateName: "test-runtimeclass-runtimeclass-1166-preconfigured-handler-",
                    Namespace: "runtimeclass-1166",
                    SelfLink: "",
                    UID: "f945b4fa-66c5-450d-a4e0-32c238d1b916",
                    ResourceVersion: "40660",
                    Generation: 0,
                    CreationTimestamp: {
                        Time: {
                            wall: 0,
                            ext: 63810487158,
                            loc: {
                                name: "Local",
                                zone: [
                                    {name: "UTC", offset: 0, isDST: false},
                                ],
                                tx: [
                                    {
                                        when: -9223372036854775808,
                                        index: 0,
                                        isstd: false,
                                        isutc: false,
                                    },
                                ],
                                extend: "UTC0",
                                cacheStart: 9223372036854775807,
                                cacheEnd: 9223372036854775807,
                                cacheZone: {name: "UTC", offset: 0, isDST: false},
                            },
                        },
                    },
                    DeletionTimestamp: nil,
                    DeletionGracePeriodSeconds: nil,
                    Labels: nil,
                    Annotations: nil,
                    OwnerReferences: nil,
                    Finalizers: nil,
                    ManagedFields: [
                        {
                            Manager: "e2e.test",
                            Operation: "Update",
                            APIVersion: "v1",
                            Time: {
                                Time: {
                                    wall: 0,
                                    ext: 63810487158,
                                    loc: {
                                        name: "Local",
                                        zone: [...],
                                        tx: [...],
                                        extend: "UTC0",
                                        cacheStart: 9223372036854775807,
                                        cacheEnd: 9223372036854775807,
                                        cacheZone: {name: ..., offset: ..., isDST: ...},
                                    },
                                },
                            },
                            FieldsType: "FieldsV1",
                            FieldsV1: {
                                Raw: "{\"f:metadata\":{\"f:generateName\":{}},\"f:spec\":{\"f:automountServiceAccountToken\":{},\"f:containers\":{\"k:{\\\"name\\\":\\\"test\\\"}\":{\".\":{},\"f:command\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:runtimeClassName\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:terminationGracePeriodSeconds\":{}}}",
                            },
                            Subresource: "",
                        },
                        {
                            Manager: "kubelet",
                            Operation: "Update",
                            APIVersion: "v1",
                            Time: {
                                Time: {
                                    wall: 0,
                                    ext: 63810487169,
                                    loc: {
                                        name: "Local",
                                        zone: [...],
                                        tx: [...],
                                        extend: "UTC0",
                                        cacheStart: 9223372036854775807,
                                        cacheEnd: 9223372036854775807,
                                        cacheZone: {name: ..., offset: ..., isDST: ...},
                                    },
                                },
                            },
                            FieldsType: "FieldsV1",
                            FieldsV1: {
                                Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}",
                            },
                            Subresource: "status",
                        },
                    ],
                },
                Spec: {
                    Volumes: nil,
                    InitContainers: nil,
                    Containers: [
                        {
                            Name: "test",
                            Image: "registry.k8s.io/e2e-test-images/busybox:1.29-4",
                            Command: ["true"],
                            Args: nil,
                            WorkingDir: "",
                            Ports: nil,
                            EnvFrom: nil,
                            Env: nil,
                            Resources: {Limits: nil, Requests: nil, Claims: nil},
                            VolumeMounts: nil,
                            VolumeDevices: nil,
                            LivenessProbe: nil,
                            ReadinessProbe: nil,
                            StartupProbe: nil,
                            Lifecycle: nil,
                            TerminationMessagePath: "/dev/termination-log",
                            TerminationMessagePolicy: "File",
                            ImagePullPolicy: "IfNotPresent",
                            SecurityContext: nil,
                            Stdin: false,
                            StdinOnce: false,
                            TTY: false,
                        },
                    ],
                    EphemeralContainers: nil,
                    RestartPolicy: "Never",
                    TerminationGracePeriodSeconds: 30,
                    ActiveDeadlineSeconds: nil,
                    DNSPolicy: "ClusterFirst",
                    NodeSelector: nil,
                    ServiceAccountName: "default",
                    DeprecatedServiceAccount: "default",
                    AutomountServiceAccountToken: false,
                    NodeName: "i-0f637d6210a139b70",
                    HostNetwork: false,
                    HostPID: false,
                    HostIPC: false,
                    ShareProcessNamespace: nil,
                    SecurityContext: {
                        SELinuxOptions: nil,
                        WindowsOptions: nil,
                        RunAsUser: nil,
                        RunAsGroup: nil,
                        RunAsNonRoot: nil,
                        SupplementalGroups: nil,
                        FSGroup: nil,
                        Sysctls: nil,
                        FSGroupChangePolicy: nil,
               ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

[FAILED] timed out while waiting for pod runtimeclass-1166/test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c to be Succeeded or Failed
In [It] at: test/e2e/common/node/runtimeclass.go:386 @ 01/28/23 07:24:18.143
< Exit [It] should run a Pod requesting a RuntimeClass with a configured handler [NodeFeature:RuntimeHandler] - test/e2e/common/node/runtimeclass.go:85 @ 01/28/23 07:24:18.143 (5m2.591s)
> Enter [AfterEach] [sig-node] RuntimeClass - test/e2e/framework/node/init/init.go:33 @ 01/28/23 07:24:18.143
Jan 28 07:24:18.143: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
< Exit [AfterEach] [sig-node] RuntimeClass - test/e2e/framework/node/init/init.go:33 @ 01/28/23 07:24:18.175 (33ms)
> Enter [DeferCleanup (Each)] [sig-node] RuntimeClass - test/e2e/common/node/runtimeclass.go:91 @ 01/28/23 07:24:18.175
< Exit [DeferCleanup (Each)] [sig-node] RuntimeClass - test/e2e/common/node/runtimeclass.go:91 @ 01/28/23 07:24:18.209 (33ms)
> Enter [DeferCleanup (Each)] [sig-node] RuntimeClass - test/e2e/framework/node/runtimeclass/runtimeclass.go:64 @ 01/28/23 07:24:18.209
STEP: Deleting pod hostexec-i-0ab74bbb764104081-dtt48 in namespace runtimeclass-1166 - test/e2e/framework/pod/delete.go:41 @ 01/28/23 07:24:18.209
< Exit [DeferCleanup (Each)] [sig-node] RuntimeClass - test/e2e/framework/node/runtimeclass/runtimeclass.go:64 @ 01/28/23 07:24:18.247 (39ms)
> Enter [DeferCleanup (Each)] [sig-node] RuntimeClass - test/e2e/framework/metrics/init/init.go:35 @ 01/28/23 07:24:18.247
< Exit [DeferCleanup (Each)] [sig-node] RuntimeClass - test/e2e/framework/metrics/init/init.go:35 @ 01/28/23 07:24:18.247 (0s)
> Enter [DeferCleanup (Each)] [sig-node] RuntimeClass - dump namespaces | framework.go:206 @ 01/28/23 07:24:18.247
STEP: dump namespace information after failure - test/e2e/framework/framework.go:284 @ 01/28/23 07:24:18.247
STEP: Collecting events from namespace "runtimeclass-1166". - test/e2e/framework/debug/dump.go:42 @ 01/28/23 07:24:18.247
STEP: Found 7 events. - test/e2e/framework/debug/dump.go:46 @ 01/28/23 07:24:18.279
Jan 28 07:24:18.279: INFO: At 2023-01-28 07:19:15 +0000 UTC - event for hostexec-i-0ab74bbb764104081-dtt48: {default-scheduler } Scheduled: Successfully assigned runtimeclass-1166/hostexec-i-0ab74bbb764104081-dtt48 to i-0ab74bbb764104081
Jan 28 07:24:18.279: INFO: At 2023-01-28 07:19:16 +0000 UTC - event for hostexec-i-0ab74bbb764104081-dtt48: {kubelet i-0ab74bbb764104081} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Jan 28 07:24:18.279: INFO: At 2023-01-28 07:19:16 +0000 UTC - event for hostexec-i-0ab74bbb764104081-dtt48: {kubelet i-0ab74bbb764104081} Created: Created container agnhost-container
Jan 28 07:24:18.279: INFO: At 2023-01-28 07:19:16 +0000 UTC - event for hostexec-i-0ab74bbb764104081-dtt48: {kubelet i-0ab74bbb764104081} Started: Started container agnhost-container
Jan 28 07:24:18.279: INFO: At 2023-01-28 07:19:18 +0000 UTC - event for test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c: {default-scheduler } Scheduled: Successfully assigned runtimeclass-1166/test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c to i-0f637d6210a139b70
Jan 28 07:24:18.279: INFO: At 2023-01-28 07:19:18 +0000 UTC - event for test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c: {kubelet i-0f637d6210a139b70} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to get sandbox runtime: no runtime for "test-handler" is configured
Jan 28 07:24:18.279: INFO: At 2023-01-28 07:24:18 +0000 UTC - event for hostexec-i-0ab74bbb764104081-dtt48: {kubelet i-0ab74bbb764104081} Killing: Stopping container agnhost-container
Jan 28 07:24:18.310: INFO: POD                                                              NODE                 PHASE    GRACE  CONDITIONS
Jan 28 07:24:18.310: INFO: test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c  i-0f637d6210a139b70  Pending         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-28 07:19:18 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-28 07:19:18 +0000 UTC ContainersNotReady containers with unready status: [test]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-28 07:19:18 +0000 UTC ContainersNotReady containers with unready status: [test]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-28 07:19:18 +0000 UTC  }]
Jan 28 07:24:18.310: INFO: 
Jan 28 07:24:18.343: INFO: Unable to fetch runtimeclass-1166/test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c/test logs: the server rejected our request for an unknown reason (get pods test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c)
Jan 28 07:24:18.376: INFO: 
Logging node info for node i-03840b5a70dc4c0e3
Jan 28 07:24:18.408: INFO: Node Info: &Node{ObjectMeta:{i-03840b5a70dc4c0e3    07b808d1-06d7-4b70-9ca2-ffe0f64e46ec 49357 0 2023-01-28 07:02:12 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:arm64 beta.kubernetes.io/instance-type:t4g.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:arm64 kubernetes.io/hostname:i-03840b5a70dc4c0e3 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t4g.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-03840b5a70dc4c0e3 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-03840b5a70dc4c0e3"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2023-01-28 07:23:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-03840b5a70dc4c0e3,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49756430336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{4027252736 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44780787229 0} {<nil>} 44780787229 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{3922395136 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-28 07:22:58 +0000 UTC,LastTransitionTime:2023-01-28 07:02:12 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-28 07:22:58 +0000 UTC,LastTransitionTime:2023-01-28 07:02:12 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-28 07:22:58 +0000 UTC,LastTransitionTime:2023-01-28 07:02:12 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-28 07:22:58 +0000 UTC,LastTransitionTime:2023-01-28 07:02:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.32.174,},NodeAddress{Type:ExternalIP,Address:3.96.175.228,},NodeAddress{Type:InternalDNS,Address:i-03840b5a70dc4c0e3.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-03840b5a70dc4c0e3.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-175-228.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21d95d2eb5817b39aed905c39e3faa,SystemUUID:ec21d95d-2eb5-817b-39ae-d905c39e3faa,BootID:c0022840-1246-459b-bca8-c2565d451680,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 22.04.1 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,KubeProxyVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,OperatingSystem:linux,Architecture:arm64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:157636062,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:116764683,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:80539316,},ContainerImage{Names:[registry.k8s.io/kube-proxy-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:63181512,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:49808364,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:47953400,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41528818,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40343601,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:28932904,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:23945474,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:23857462,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:22678632,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:22434180,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:22394501,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:18561972,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18267611,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:8502345,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8223819,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:7685305,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6860089,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:5025027,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:773367,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:772933,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:268051,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:253553,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 28 07:24:18.408: INFO: 
Logging kubelet events for node i-03840b5a70dc4c0e3
Jan 28 07:24:18.443: INFO: 
Logging pods the kubelet thinks is on node i-03840b5a70dc4c0e3
Jan 28 07:24:18.478: INFO: cilium-55gmk started at 2023-01-28 07:02:13 +0000 UTC (1+1 container statuses recorded)
Jan 28 07:24:18.478: INFO: 	Init container clean-cilium-state ready: true, restart count 0
Jan 28 07:24:18.478: INFO: 	Container cilium-agent ready: true, restart count 0
Jan 28 07:24:18.478: INFO: ebs-csi-node-g8bxd started at 2023-01-28 07:02:13 +0000 UTC (0+3 container statuses recorded)
Jan 28 07:24:18.478: INFO: 	Container ebs-plugin ready: true, restart count 0
Jan 28 07:24:18.478: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:24:18.478: INFO: 	Container node-driver-registrar ready: true, restart count 0
Jan 28 07:24:18.655: INFO: 
Latency metrics for node i-03840b5a70dc4c0e3
Jan 28 07:24:18.655: INFO: 
Logging node info for node i-05f48d9d08fa62714
Jan 28 07:24:18.689: INFO: Node Info: &Node{ObjectMeta:{i-05f48d9d08fa62714    9e0872b7-c7ff-4d52-8535-3a3f499cab86 49884 0 2023-01-28 07:02:13 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:arm64 beta.kubernetes.io/instance-type:t4g.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:arm64 kubernetes.io/hostname:i-05f48d9d08fa62714 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t4g.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-05f48d9d08fa62714 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-05f48d9d08fa62714"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2023-01-28 07:23:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-05f48d9d08fa62714,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49756430336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{4027252736 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44780787229 0} {<nil>} 44780787229 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{3922395136 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-28 07:23:19 +0000 UTC,LastTransitionTime:2023-01-28 07:02:13 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-28 07:23:19 +0000 UTC,LastTransitionTime:2023-01-28 07:02:13 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-28 07:23:19 +0000 UTC,LastTransitionTime:2023-01-28 07:02:13 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-28 07:23:19 +0000 UTC,LastTransitionTime:2023-01-28 07:02:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.56.199,},NodeAddress{Type:ExternalIP,Address:3.96.212.120,},NodeAddress{Type:InternalDNS,Address:i-05f48d9d08fa62714.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-05f48d9d08fa62714.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-212-120.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec223aa27d20f057bcefa98a58656d96,SystemUUID:ec223aa2-7d20-f057-bcef-a98a58656d96,BootID:4b7e67df-7ab4-44b0-bb7b-1a2eb086bdad,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 22.04.1 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,KubeProxyVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,OperatingSystem:linux,Architecture:arm64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:157636062,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:116764683,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:80539316,},ContainerImage{Names:[registry.k8s.io/kube-proxy-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:63181512,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:49808364,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:47953400,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41528818,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40343601,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:28932904,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:23945474,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:22678632,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:22434180,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:22394501,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:18561972,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18267611,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:13766234,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:8502345,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8223819,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:7685305,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:6883347,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6860089,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:5025027,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:773367,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:772933,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:268051,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:253553,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 28 07:24:18.689: INFO: 
Logging kubelet events for node i-05f48d9d08fa62714
Jan 28 07:24:18.724: INFO: 
Logging pods the kubelet thinks is on node i-05f48d9d08fa62714
Jan 28 07:24:18.759: INFO: cilium-wfd8c started at 2023-01-28 07:02:14 +0000 UTC (1+1 container statuses recorded)
Jan 28 07:24:18.759: INFO: 	Init container clean-cilium-state ready: true, restart count 0
Jan 28 07:24:18.759: INFO: 	Container cilium-agent ready: true, restart count 0
Jan 28 07:24:18.759: INFO: ebs-csi-node-9tl4h started at 2023-01-28 07:02:14 +0000 UTC (0+3 container statuses recorded)
Jan 28 07:24:18.759: INFO: 	Container ebs-plugin ready: true, restart count 0
Jan 28 07:24:18.759: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:24:18.759: INFO: 	Container node-driver-registrar ready: true, restart count 0
Jan 28 07:24:18.759: INFO: coredns-559769c974-d52jw started at 2023-01-28 07:02:50 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:24:18.759: INFO: 	Container coredns ready: true, restart count 0
Jan 28 07:24:18.960: INFO: 
Latency metrics for node i-05f48d9d08fa62714
Jan 28 07:24:18.960: INFO: 
Logging node info for node i-0ab74bbb764104081
Jan 28 07:24:18.992: INFO: Node Info: &Node{ObjectMeta:{i-0ab74bbb764104081    96aa74ac-06e1-445f-9738-5fa3715ba0be 49664 0 2023-01-28 07:02:11 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:arm64 beta.kubernetes.io/instance-type:t4g.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:arm64 kubernetes.io/hostname:i-0ab74bbb764104081 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t4g.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0ab74bbb764104081 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-expansion-4070":"i-0ab74bbb764104081","ebs.csi.aws.com":"i-0ab74bbb764104081"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-01-28 07:02:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-28 07:02:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-28 07:23:16 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-28 07:23:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0ab74bbb764104081,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49756430336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{4027248640 0} {<nil>} 3932860Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44780787229 0} {<nil>} 44780787229 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{3922391040 0} {<nil>} 3830460Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-28 07:23:16 +0000 UTC,LastTransitionTime:2023-01-28 07:02:11 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-28 07:23:16 +0000 UTC,LastTransitionTime:2023-01-28 07:02:11 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-28 07:23:16 +0000 UTC,LastTransitionTime:2023-01-28 07:02:11 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-28 07:23:16 +0000 UTC,LastTransitionTime:2023-01-28 07:02:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.17,},NodeAddress{Type:ExternalIP,Address:35.183.243.152,},NodeAddress{Type:InternalDNS,Address:i-0ab74bbb764104081.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ab74bbb764104081.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-183-243-152.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec224fe09fc7d0b0236a5ba8f22f67e1,SystemUUID:ec224fe0-9fc7-d0b0-236a-5ba8f22f67e1,BootID:cf011503-4fe6-4c28-8708-a19a9015457c,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 22.04.1 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,KubeProxyVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,OperatingSystem:linux,Architecture:arm64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:157636062,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:116764683,},ContainerImage{Names:[registry.k8s.io/kube-proxy-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:63181512,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:49808364,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:47953400,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41528818,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40343601,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:28932904,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:23945474,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:22678632,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:22434180,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:22394501,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:21735644,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:21331383,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:20848413,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:20565480,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:19007623,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18267611,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:13766234,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:8502345,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8223819,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:7685305,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:6883347,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6860089,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:5025027,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2003505,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:773367,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:772933,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:268051,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:253553,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-4070^960c9d7b-9edc-11ed-8160-42205b68a4d1],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-expansion-4070^960c9d7b-9edc-11ed-8160-42205b68a4d1,DevicePath:,},},Config:nil,},}
Jan 28 07:24:18.992: INFO: 
Logging kubelet events for node i-0ab74bbb764104081
Jan 28 07:24:19.027: INFO: 
Logging pods the kubelet thinks is on node i-0ab74bbb764104081
Jan 28 07:24:19.064: INFO: csi-mockplugin-attacher-0 started at 2023-01-28 07:22:46 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:24:19.064: INFO: 	Container csi-attacher ready: true, restart count 0
Jan 28 07:24:19.064: INFO: ebs-csi-controller-555658678f-dfmf4 started at 2023-01-28 07:02:22 +0000 UTC (0+5 container statuses recorded)
Jan 28 07:24:19.064: INFO: 	Container csi-attacher ready: true, restart count 0
Jan 28 07:24:19.064: INFO: 	Container csi-provisioner ready: true, restart count 0
Jan 28 07:24:19.065: INFO: 	Container csi-resizer ready: true, restart count 0
Jan 28 07:24:19.065: INFO: 	Container ebs-plugin ready: true, restart count 0
Jan 28 07:24:19.065: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:24:19.065: INFO: ebs-csi-node-zkczw started at 2023-01-28 07:02:12 +0000 UTC (0+3 container statuses recorded)
Jan 28 07:24:19.065: INFO: 	Container ebs-plugin ready: true, restart count 0
Jan 28 07:24:19.065: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:24:19.065: INFO: 	Container node-driver-registrar ready: true, restart count 0
Jan 28 07:24:19.065: INFO: csi-mockplugin-0 started at 2023-01-28 07:22:46 +0000 UTC (0+3 container statuses recorded)
Jan 28 07:24:19.065: INFO: 	Container csi-provisioner ready: true, restart count 0
Jan 28 07:24:19.065: INFO: 	Container driver-registrar ready: true, restart count 0
Jan 28 07:24:19.065: INFO: 	Container mock ready: true, restart count 0
Jan 28 07:24:19.065: INFO: pvc-volume-tester-nndbb started at 2023-01-28 07:22:58 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:24:19.065: INFO: 	Container volume-tester ready: true, restart count 0
Jan 28 07:24:19.065: INFO: cilium-lfmv7 started at 2023-01-28 07:02:12 +0000 UTC (1+1 container statuses recorded)
Jan 28 07:24:19.065: INFO: 	Init container clean-cilium-state ready: true, restart count 0
Jan 28 07:24:19.065: INFO: 	Container cilium-agent ready: true, restart count 0
Jan 28 07:24:19.065: INFO: coredns-559769c974-fnksb started at 2023-01-28 07:02:22 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:24:19.065: INFO: 	Container coredns ready: true, restart count 0
Jan 28 07:24:19.065: INFO: coredns-autoscaler-7cb5c5b969-lhx5j started at 2023-01-28 07:02:22 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:24:19.065: INFO: 	Container autoscaler ready: true, restart count 0
Jan 28 07:24:19.266: INFO: 
Latency metrics for node i-0ab74bbb764104081
Jan 28 07:24:19.266: INFO: 
Logging node info for node i-0c0709f8c7952ec00
Jan 28 07:24:19.298: INFO: Node Info: &Node{ObjectMeta:{i-0c0709f8c7952ec00    ee8d889d-42fa-49b7-9c7f-f8b5d1a036f5 49076 0 2023-01-28 06:59:47 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:arm64 beta.kubernetes.io/instance-type:c6g.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:arm64 kubernetes.io/hostname:i-0c0709f8c7952ec00 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c6g.large topology.ebs.csi.aws.com/zone:ca-central-1a topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0c0709f8c7952ec00"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-28 06:59:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-28 07:00:14 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-28 07:01:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-28 07:01:06 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-28 07:01:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-28 07:22:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0c0709f8c7952ec00,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49756430336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{4002131968 0} {<nil>} 3908332Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44780787229 0} {<nil>} 44780787229 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{3897274368 0} {<nil>} 3805932Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-28 07:22:55 +0000 UTC,LastTransitionTime:2023-01-28 06:59:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-28 07:22:55 +0000 UTC,LastTransitionTime:2023-01-28 06:59:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-28 07:22:55 +0000 UTC,LastTransitionTime:2023-01-28 06:59:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-28 07:22:55 +0000 UTC,LastTransitionTime:2023-01-28 07:01:08 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.56.68,},NodeAddress{Type:ExternalIP,Address:99.79.32.119,},NodeAddress{Type:InternalDNS,Address:i-0c0709f8c7952ec00.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0c0709f8c7952ec00.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-99-79-32-119.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a6c102ceaf46a9a6436a04797b985,SystemUUID:ec2a6c10-2cea-f46a-9a64-36a04797b985,BootID:32e68828-ba43-463f-b849-18b0db770fe8,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 22.04.1 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,KubeProxyVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,OperatingSystem:linux,Architecture:arm64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:157797353,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:157636062,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:151677080,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:134531559,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:131348942,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:126979107,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:121190708,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:81436097,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:81120118,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:81114891,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:80539316,},ContainerImage{Names:[registry.k8s.io/kube-proxy-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:63181512,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:61500433,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:60827222,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:59553904,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:56048433,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:41190270,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:39793987,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:28932904,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:24567287,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:18345226,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:8502345,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:7685305,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4691073,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:253553,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 28 07:24:19.298: INFO: 
Logging kubelet events for node i-0c0709f8c7952ec00
Jan 28 07:24:19.333: INFO: 
Logging pods the kubelet thinks is on node i-0c0709f8c7952ec00
Jan 28 07:24:19.383: INFO: kube-scheduler-i-0c0709f8c7952ec00 started at 2023-01-28 06:58:21 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:24:19.383: INFO: 	Container kube-scheduler ready: true, restart count 0
Jan 28 07:24:19.383: INFO: etcd-manager-events-i-0c0709f8c7952ec00 started at 2023-01-28 06:58:21 +0000 UTC (10+1 container statuses recorded)
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Container etcd-manager ready: true, restart count 0
Jan 28 07:24:19.383: INFO: etcd-manager-main-i-0c0709f8c7952ec00 started at 2023-01-28 06:58:21 +0000 UTC (10+1 container statuses recorded)
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Container etcd-manager ready: true, restart count 0
Jan 28 07:24:19.383: INFO: kube-apiserver-i-0c0709f8c7952ec00 started at 2023-01-28 06:58:21 +0000 UTC (0+2 container statuses recorded)
Jan 28 07:24:19.383: INFO: 	Container healthcheck ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Container kube-apiserver ready: true, restart count 2
Jan 28 07:24:19.383: INFO: kube-controller-manager-i-0c0709f8c7952ec00 started at 2023-01-28 06:58:21 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:24:19.383: INFO: 	Container kube-controller-manager ready: true, restart count 4
Jan 28 07:24:19.383: INFO: ebs-csi-node-lgtrp started at 2023-01-28 07:00:49 +0000 UTC (0+3 container statuses recorded)
Jan 28 07:24:19.383: INFO: 	Container ebs-plugin ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Container node-driver-registrar ready: true, restart count 0
Jan 28 07:24:19.383: INFO: aws-cloud-controller-manager-wzlg2 started at 2023-01-28 07:00:49 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:24:19.383: INFO: 	Container aws-cloud-controller-manager ready: true, restart count 0
Jan 28 07:24:19.383: INFO: cilium-w9t7g started at 2023-01-28 07:00:49 +0000 UTC (1+1 container statuses recorded)
Jan 28 07:24:19.383: INFO: 	Init container clean-cilium-state ready: true, restart count 0
Jan 28 07:24:19.383: INFO: 	Container cilium-agent ready: true, restart count 0
Jan 28 07:24:19.383: INFO: kops-controller-sl7df started at 2023-01-28 07:00:49 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:24:19.383: INFO: 	Container kops-controller ready: true, restart count 0
Jan 28 07:24:19.383: INFO: cilium-operator-6f7d7f696f-qd5rh started at 2023-01-28 07:00:50 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:24:19.383: INFO: 	Container cilium-operator ready: true, restart count 0
Jan 28 07:24:19.383: INFO: dns-controller-85d56fc6db-brmzv started at 2023-01-28 07:00:50 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:24:19.383: INFO: 	Container dns-controller ready: true, restart count 0
Jan 28 07:24:19.542: INFO: 
Latency metrics for node i-0c0709f8c7952ec00
Jan 28 07:24:19.542: INFO: 
Logging node info for node i-0f637d6210a139b70
Jan 28 07:24:19.574: INFO: Node Info: &Node{ObjectMeta:{i-0f637d6210a139b70    e6187f2f-df4a-409d-84ab-b2e2ad79b8cf 45385 0 2023-01-28 07:02:13 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:arm64 beta.kubernetes.io/instance-type:t4g.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:arm64 kubernetes.io/hostname:i-0f637d6210a139b70 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t4g.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0f637d6210a139b70 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0f637d6210a139b70"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kubelet Update v1 2023-01-28 07:21:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0f637d6210a139b70,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49756430336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{4027248640 0} {<nil>} 3932860Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44780787229 0} {<nil>} 44780787229 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{3922391040 0} {<nil>} 3830460Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-28 07:21:28 +0000 UTC,LastTransitionTime:2023-01-28 07:02:13 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-28 07:21:28 +0000 UTC,LastTransitionTime:2023-01-28 07:02:13 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-28 07:21:28 +0000 UTC,LastTransitionTime:2023-01-28 07:02:13 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-28 07:21:28 +0000 UTC,LastTransitionTime:2023-01-28 07:02:23 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.154,},NodeAddress{Type:ExternalIP,Address:3.99.165.197,},NodeAddress{Type:InternalDNS,Address:i-0f637d6210a139b70.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f637d6210a139b70.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-99-165-197.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dcb89700752b2acd2ab3f959203ae,SystemUUID:ec2dcb89-7007-52b2-acd2-ab3f959203ae,BootID:8643c9bc-2d46-4dd1-b176-7c7d8439718a,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 22.04.1 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,KubeProxyVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,OperatingSystem:linux,Architecture:arm64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:157636062,},ContainerImage{Names:[registry.k8s.io/kube-proxy-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:63181512,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:49808364,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41528818,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40343601,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:28932904,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:23945474,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:22678632,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:22434180,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:22394501,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:21735644,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:20848413,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:20565480,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18267611,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:8502345,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8223819,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:7685305,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6860089,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:772933,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:268051,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:253553,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 28 07:24:19.574: INFO: 
Logging kubelet events for node i-0f637d6210a139b70
Jan 28 07:24:19.609: INFO: 
Logging pods the kubelet thinks is on node i-0f637d6210a139b70
Jan 28 07:24:19.644: INFO: test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c started at 2023-01-28 07:19:18 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:24:19.644: INFO: 	Container test ready: false, restart count 0
Jan 28 07:24:19.644: INFO: cilium-lmxnd started at 2023-01-28 07:02:14 +0000 UTC (1+1 container statuses recorded)
Jan 28 07:24:19.644: INFO: 	Init container clean-cilium-state ready: true, restart count 0
Jan 28 07:24:19.644: INFO: 	Container cilium-agent ready: true, restart count 0
Jan 28 07:24:19.644: INFO: ebs-csi-node-28h7q started at 2023-01-28 07:02:14 +0000 UTC (0+3 container statuses recorded)
Jan 28 07:24:19.644: INFO: 	Container ebs-plugin ready: true, restart count 0
Jan 28 07:24:19.644: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:24:19.644: INFO: 	Container node-driver-registrar ready: true, restart count 0
Jan 28 07:24:19.644: INFO: ebs-csi-controller-555658678f-vn4s8 started at 2023-01-28 07:02:24 +0000 UTC (0+5 container statuses recorded)
Jan 28 07:24:19.644: INFO: 	Container csi-attacher ready: true, restart count 0
Jan 28 07:24:19.644: INFO: 	Container csi-provisioner ready: true, restart count 0
Jan 28 07:24:19.644: INFO: 	Container csi-resizer ready: true, restart count 0
Jan 28 07:24:19.644: INFO: 	Container ebs-plugin ready: true, restart count 0
Jan 28 07:24:19.644: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:24:19.834: INFO: 
Latency metrics for node i-0f637d6210a139b70
END STEP: dump namespace information after failure - test/e2e/framework/framework.go:284 @ 01/28/23 07:24:19.834 (1.586s)
< Exit [DeferCleanup (Each)] [sig-node] RuntimeClass - dump namespaces | framework.go:206 @ 01/28/23 07:24:19.834 (1.586s)
> Enter [DeferCleanup (Each)] [sig-node] RuntimeClass - tear down framework | framework.go:203 @ 01/28/23 07:24:19.834
STEP: Destroying namespace "runtimeclass-1166" for this suite. - test/e2e/framework/framework.go:347 @ 01/28/23 07:24:19.834
< Exit [DeferCleanup (Each)] [sig-node] RuntimeClass - tear down framework | framework.go:203 @ 01/28/23 07:24:19.868 (34ms)
> Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 01/28/23 07:24:19.868
< Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 01/28/23 07:24:19.868 (0s)

				Click to see stdout/stderrfrom junit_01.xml

Find runtimeclass-1166/test-runtimeclass-runtimeclass-1166-preconfigured-handler-nbw8c mentions in log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-node] RuntimeClass should run a Pod requesting a RuntimeClass with scheduling without taints 5m24s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-node\]\sRuntimeClass\sshould\srun\sa\sPod\srequesting\sa\sRuntimeClass\swith\sscheduling\swithout\staints$'

[FAILED] timed out while waiting for pod runtimeclass-3956/test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6 to be not pending
In [It] at: test/e2e/node/runtimeclass.go:167 @ 01/28/23 07:10:41.453

				> Enter [BeforeEach] [sig-node] RuntimeClass - set up framework | framework.go:188 @ 01/28/23 07:05:19.526
STEP: Creating a kubernetes client - test/e2e/framework/framework.go:208 @ 01/28/23 07:05:19.526
Jan 28 07:05:19.526: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename runtimeclass - test/e2e/framework/framework.go:247 @ 01/28/23 07:05:19.528
STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:256 @ 01/28/23 07:05:19.652
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 01/28/23 07:05:19.713
< Exit [BeforeEach] [sig-node] RuntimeClass - set up framework | framework.go:188 @ 01/28/23 07:05:19.777 (251ms)
> Enter [BeforeEach] [sig-node] RuntimeClass - test/e2e/framework/metrics/init/init.go:33 @ 01/28/23 07:05:19.777
< Exit [BeforeEach] [sig-node] RuntimeClass - test/e2e/framework/metrics/init/init.go:33 @ 01/28/23 07:05:19.777 (0s)
> Enter [It] should run a Pod requesting a RuntimeClass with scheduling without taints  - test/e2e/node/runtimeclass.go:131 @ 01/28/23 07:05:19.777
Jan 28 07:05:19.914: INFO: Waiting up to 5m0s for pod "hostexec-i-03840b5a70dc4c0e3-t5zws" in namespace "runtimeclass-3956" to be "running"
Jan 28 07:05:19.956: INFO: Pod "hostexec-i-03840b5a70dc4c0e3-t5zws": Phase="Pending", Reason="", readiness=false. Elapsed: 41.506838ms
Jan 28 07:05:21.988: INFO: Pod "hostexec-i-03840b5a70dc4c0e3-t5zws": Phase="Pending", Reason="", readiness=false. Elapsed: 2.073455633s
Jan 28 07:05:23.993: INFO: Pod "hostexec-i-03840b5a70dc4c0e3-t5zws": Phase="Pending", Reason="", readiness=false. Elapsed: 4.078051167s
Jan 28 07:05:25.993: INFO: Pod "hostexec-i-03840b5a70dc4c0e3-t5zws": Phase="Pending", Reason="", readiness=false. Elapsed: 6.078664021s
Jan 28 07:05:27.989: INFO: Pod "hostexec-i-03840b5a70dc4c0e3-t5zws": Phase="Running", Reason="", readiness=true. Elapsed: 8.074117897s
Jan 28 07:05:27.989: INFO: Pod "hostexec-i-03840b5a70dc4c0e3-t5zws" satisfied condition "running"
Jan 28 07:05:27.989: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c if [ -e '/etc/containerd/config.toml' ]; then
grep -q 'runtimes.test-handler' /etc/containerd/config.toml
  return
fi

if [ -e '/etc/crio/crio.conf' ]; then
  grep -q 'runtimes.test-handler' /etc/crio/crio.conf
  return
fi
] Namespace:runtimeclass-3956 PodName:hostexec-i-03840b5a70dc4c0e3-t5zws ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true Quiet:false}
Jan 28 07:05:27.989: INFO: >>> kubeConfig: /root/.kube/config
Jan 28 07:05:27.989: INFO: ExecWithOptions: Clientset creation
Jan 28 07:05:27.989: INFO: ExecWithOptions: execute(POST https://api-e2e-e2e-kops-warm-poo-agujp3-f36db33deca21475.elb.ca-central-1.amazonaws.com/api/v1/namespaces/runtimeclass-3956/pods/hostexec-i-03840b5a70dc4c0e3-t5zws/exec?command=nsenter&command=--mount%3D%2Frootfs%2Fproc%2F1%2Fns%2Fmnt&command=--&command=sh&command=-c&command=if+%5B+-e+%27%2Fetc%2Fcontainerd%2Fconfig.toml%27+%5D%3B+then%0Agrep+-q+%27runtimes.test-handler%27+%2Fetc%2Fcontainerd%2Fconfig.toml%0A++return%0Afi%0A%0Aif+%5B+-e+%27%2Fetc%2Fcrio%2Fcrio.conf%27+%5D%3B+then%0A++grep+-q+%27runtimes.test-handler%27+%2Fetc%2Fcrio%2Fcrio.conf%0A++return%0Afi%0A&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true)
STEP: Trying to launch a pod without a label to get a node which can launch it. - test/e2e/scheduling/predicates.go:1042 @ 01/28/23 07:05:28.971
Jan 28 07:05:29.008: INFO: Waiting up to 1m0s for pod "without-label" in namespace "runtimeclass-3956" to be "running"
Jan 28 07:05:29.040: INFO: Pod "without-label": Phase="Pending", Reason="", readiness=false. Elapsed: 31.310635ms
Jan 28 07:05:31.072: INFO: Pod "without-label": Phase="Pending", Reason="", readiness=false. Elapsed: 2.063594553s
Jan 28 07:05:33.086: INFO: Pod "without-label": Phase="Pending", Reason="", readiness=false. Elapsed: 4.077564599s
Jan 28 07:05:35.072: INFO: Pod "without-label": Phase="Pending", Reason="", readiness=false. Elapsed: 6.063599257s
Jan 28 07:05:37.072: INFO: Pod "without-label": Phase="Pending", Reason="", readiness=false. Elapsed: 8.063829509s
Jan 28 07:05:39.072: INFO: Pod "without-label": Phase="Pending", Reason="", readiness=false. Elapsed: 10.063468568s
Jan 28 07:05:41.072: INFO: Pod "without-label": Phase="Running", Reason="", readiness=true. Elapsed: 12.063586849s
Jan 28 07:05:41.072: INFO: Pod "without-label" satisfied condition "running"
STEP: Explicitly delete pod here to free the resource it takes. - test/e2e/scheduling/predicates.go:973 @ 01/28/23 07:05:41.104
STEP: Trying to apply a label on the found node. - test/e2e/node/runtimeclass.go:148 @ 01/28/23 07:05:41.151
STEP: verifying the node has the label foo-ca6146d2-85d4-4b06-a635-605b2d6f048f bar - test/e2e/framework/node/helper.go:63 @ 01/28/23 07:05:41.188
STEP: verifying the node has the label fizz-54d7a2c1-2a70-4c82-a7b4-3fee7394e883 buzz - test/e2e/framework/node/helper.go:63 @ 01/28/23 07:05:41.259
STEP: Trying to create runtimeclass and pod - test/e2e/node/runtimeclass.go:155 @ 01/28/23 07:05:41.29
Jan 28 07:05:41.356: INFO: Waiting up to 5m0s for pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6" in namespace "runtimeclass-3956" to be "not pending"
Jan 28 07:05:41.388: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 31.377278ms
Jan 28 07:05:43.422: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2.065845333s
Jan 28 07:05:45.422: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4.065273332s
Jan 28 07:05:47.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 6.063403362s
Jan 28 07:05:49.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 8.063348001s
Jan 28 07:05:51.424: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 10.067652133s
Jan 28 07:05:53.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 12.06415028s
Jan 28 07:05:55.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 14.063955908s
Jan 28 07:05:57.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 16.063910267s
Jan 28 07:05:59.422: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 18.065321825s
Jan 28 07:06:01.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 20.064276201s
Jan 28 07:06:03.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 22.064203427s
Jan 28 07:06:05.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 24.065119794s
Jan 28 07:06:07.423: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 26.066715186s
Jan 28 07:06:09.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 28.064966068s
Jan 28 07:06:11.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 30.064703487s
Jan 28 07:06:13.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 32.063803913s
Jan 28 07:06:15.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 34.063928365s
Jan 28 07:06:17.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 36.063645097s
Jan 28 07:06:19.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 38.063403723s
Jan 28 07:06:21.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 40.064057646s
Jan 28 07:06:23.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 42.064230625s
Jan 28 07:06:25.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 44.063578457s
Jan 28 07:06:27.423: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 46.066288635s
Jan 28 07:06:29.423: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 48.066584914s
Jan 28 07:06:31.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 50.063889497s
Jan 28 07:06:33.423: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 52.066345014s
Jan 28 07:06:35.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 54.064037848s
Jan 28 07:06:37.422: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 56.065563169s
Jan 28 07:06:39.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 58.063685976s
Jan 28 07:06:41.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.063462709s
Jan 28 07:06:43.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.064021351s
Jan 28 07:06:45.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.06339697s
Jan 28 07:06:47.427: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.07020684s
Jan 28 07:06:49.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.063801034s
Jan 28 07:06:51.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.064251228s
Jan 28 07:06:53.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.063532254s
Jan 28 07:06:55.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.064345257s
Jan 28 07:06:57.422: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.065463292s
Jan 28 07:06:59.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.0644461s
Jan 28 07:07:01.426: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.069419678s
Jan 28 07:07:03.422: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.066109578s
Jan 28 07:07:05.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.063215563s
Jan 28 07:07:07.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.06497544s
Jan 28 07:07:09.424: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.068041924s
Jan 28 07:07:11.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.063701198s
Jan 28 07:07:13.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.064352466s
Jan 28 07:07:15.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.063812632s
Jan 28 07:07:17.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.063913253s
Jan 28 07:07:19.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.064475602s
Jan 28 07:07:21.422: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.065232753s
Jan 28 07:07:23.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.064446209s
Jan 28 07:07:25.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.063631962s
Jan 28 07:07:27.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.06416964s
Jan 28 07:07:29.423: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.066306723s
Jan 28 07:07:31.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.063748405s
Jan 28 07:07:33.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.064375063s
Jan 28 07:07:35.424: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.067217588s
Jan 28 07:07:37.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.064354666s
Jan 28 07:07:39.424: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.067170757s
Jan 28 07:07:41.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.064286213s
Jan 28 07:07:43.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.064722229s
Jan 28 07:07:45.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.063886288s
Jan 28 07:07:47.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.064059171s
Jan 28 07:07:49.428: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.071287568s
Jan 28 07:07:51.423: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.067085964s
Jan 28 07:07:53.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.063532104s
Jan 28 07:07:55.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.065128083s
Jan 28 07:07:57.425: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.06880542s
Jan 28 07:07:59.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.064721828s
Jan 28 07:08:01.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.0650939s
Jan 28 07:08:03.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.063803084s
Jan 28 07:08:05.424: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.067685576s
Jan 28 07:08:07.422: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.065182608s
Jan 28 07:08:09.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.064355189s
Jan 28 07:08:11.424: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.067181835s
Jan 28 07:08:13.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.064007988s
Jan 28 07:08:15.419: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.062958379s
Jan 28 07:08:17.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.06511319s
Jan 28 07:08:19.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.065064945s
Jan 28 07:08:21.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.063283303s
Jan 28 07:08:23.447: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.090274653s
Jan 28 07:08:25.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.063894763s
Jan 28 07:08:27.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.063927513s
Jan 28 07:08:29.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.063891586s
Jan 28 07:08:31.422: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.065432513s
Jan 28 07:08:33.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.063791809s
Jan 28 07:08:35.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.064469166s
Jan 28 07:08:37.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.063894195s
Jan 28 07:08:39.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.065047769s
Jan 28 07:08:41.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.064117658s
Jan 28 07:08:43.419: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.062934335s
Jan 28 07:08:45.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.063867193s
Jan 28 07:08:47.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.064875146s
Jan 28 07:08:49.419: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.063019646s
Jan 28 07:08:51.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.063450904s
Jan 28 07:08:53.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.065009777s
Jan 28 07:08:55.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.064005768s
Jan 28 07:08:57.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.063166054s
Jan 28 07:08:59.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.064229074s
Jan 28 07:09:01.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.063442179s
Jan 28 07:09:03.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.063769346s
Jan 28 07:09:05.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.063970949s
Jan 28 07:09:07.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.063810057s
Jan 28 07:09:09.426: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.069955378s
Jan 28 07:09:11.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.063711179s
Jan 28 07:09:13.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.063780925s
Jan 28 07:09:15.419: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.062853466s
Jan 28 07:09:17.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.063820229s
Jan 28 07:09:19.422: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.065182479s
Jan 28 07:09:21.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.064485467s
Jan 28 07:09:23.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.063795588s
Jan 28 07:09:25.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.064087724s
Jan 28 07:09:27.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.064346151s
Jan 28 07:09:29.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.063745641s
Jan 28 07:09:31.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.063673115s
Jan 28 07:09:33.424: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.068063747s
Jan 28 07:09:35.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.064676242s
Jan 28 07:09:37.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.064259472s
Jan 28 07:09:39.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.063875989s
Jan 28 07:09:41.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.064120534s
Jan 28 07:09:43.422: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.06550795s
Jan 28 07:09:45.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.064016363s
Jan 28 07:09:47.427: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.070449899s
Jan 28 07:09:49.429: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.072560466s
Jan 28 07:09:51.438: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.081419593s
Jan 28 07:09:53.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.063713306s
Jan 28 07:09:55.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.064080238s
Jan 28 07:09:57.423: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.067014648s
Jan 28 07:09:59.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.06389562s
Jan 28 07:10:01.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.06442255s
Jan 28 07:10:03.425: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.068401529s
Jan 28 07:10:05.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.063987773s
Jan 28 07:10:07.429: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.072800921s
Jan 28 07:10:09.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.064028627s
Jan 28 07:10:11.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.063713448s
Jan 28 07:10:13.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.064707952s
Jan 28 07:10:15.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.063367912s
Jan 28 07:10:17.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.064173825s
Jan 28 07:10:19.425: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.068860856s
Jan 28 07:10:21.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.063909233s
Jan 28 07:10:23.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.064989239s
Jan 28 07:10:25.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.063733085s
Jan 28 07:10:27.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.063816071s
Jan 28 07:10:29.422: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.065944489s
Jan 28 07:10:31.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.063686725s
Jan 28 07:10:33.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.064085313s
Jan 28 07:10:35.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.064178355s
Jan 28 07:10:37.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.064383339s
Jan 28 07:10:39.421: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.064212621s
Jan 28 07:10:41.420: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.063861995s
Jan 28 07:10:41.452: INFO: Pod "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.095433367s
Jan 28 07:10:41.453: INFO: Unexpected error: 
    <*pod.timeoutError | 0xc001fdcf90>: {
        msg: "timed out while waiting for pod runtimeclass-3956/test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6 to be not pending",
        observedObjects: [
            <*v1.Pod | 0xc002912900>{
                TypeMeta: {Kind: "", APIVersion: ""},
                ObjectMeta: {
                    Name: "test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6",
                    GenerateName: "test-runtimeclass-runtimeclass-3956-non-conflict-runtimeclass-",
                    Namespace: "runtimeclass-3956",
                    SelfLink: "",
                    UID: "435d3f9e-6387-4d94-b08d-2ff56188958c",
                    ResourceVersion: "2811",
                    Generation: 0,
                    CreationTimestamp: {
                        Time: {
                            wall: 0,
                            ext: 63810486341,
                            loc: {
                                name: "Local",
                                zone: [
                                    {name: "UTC", offset: 0, isDST: false},
                                ],
                                tx: [
                                    {
                                        when: -9223372036854775808,
                                        index: 0,
                                        isstd: false,
                                        isutc: false,
                                    },
                                ],
                                extend: "UTC0",
                                cacheStart: 9223372036854775807,
                                cacheEnd: 9223372036854775807,
                                cacheZone: {name: "UTC", offset: 0, isDST: false},
                            },
                        },
                    },
                    DeletionTimestamp: nil,
                    DeletionGracePeriodSeconds: nil,
                    Labels: nil,
                    Annotations: nil,
                    OwnerReferences: nil,
                    Finalizers: nil,
                    ManagedFields: [
                        {
                            Manager: "e2e.test",
                            Operation: "Update",
                            APIVersion: "v1",
                            Time: {
                                Time: {
                                    wall: 0,
                                    ext: 63810486341,
                                    loc: {
                                        name: "Local",
                                        zone: [...],
                                        tx: [...],
                                        extend: "UTC0",
                                        cacheStart: 9223372036854775807,
                                        cacheEnd: 9223372036854775807,
                                        cacheZone: {name: ..., offset: ..., isDST: ...},
                                    },
                                },
                            },
                            FieldsType: "FieldsV1",
                            FieldsV1: {
                                Raw: "{\"f:metadata\":{\"f:generateName\":{}},\"f:spec\":{\"f:automountServiceAccountToken\":{},\"f:containers\":{\"k:{\\\"name\\\":\\\"test\\\"}\":{\".\":{},\"f:command\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:nodeSelector\":{},\"f:restartPolicy\":{},\"f:runtimeClassName\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:terminationGracePeriodSeconds\":{}}}",
                            },
                            Subresource: "",
                        },
                        {
                            Manager: "kubelet",
                            Operation: "Update",
                            APIVersion: "v1",
                            Time: {
                                Time: {
                                    wall: 0,
                                    ext: 63810486344,
                                    loc: {
                                        name: "Local",
                                        zone: [...],
                                        tx: [...],
                                        extend: "UTC0",
                                        cacheStart: 9223372036854775807,
                                        cacheEnd: 9223372036854775807,
                                        cacheZone: {name: ..., offset: ..., isDST: ...},
                                    },
                                },
                            },
                            FieldsType: "FieldsV1",
                            FieldsV1: {
                                Raw: "{\"f:status\":{\"f:conditions\":{\"k:{\\\"type\\\":\\\"ContainersReady\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Initialized\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Ready\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:containerStatuses\":{},\"f:hostIP\":{},\"f:startTime\":{}}}",
                            },
                            Subresource: "status",
                        },
                    ],
                },
                Spec: {
                    Volumes: nil,
                    InitContainers: nil,
                    Containers: [
                        {
                            Name: "test",
                            Image: "registry.k8s.io/e2e-test-images/busybox:1.29-4",
                            Command: ["true"],
                            Args: nil,
                            WorkingDir: "",
                            Ports: nil,
                            EnvFrom: nil,
                            Env: nil,
                            Resources: {Limits: nil, Requests: nil, Claims: nil},
                            VolumeMounts: nil,
                            VolumeDevices: nil,
                            LivenessProbe: nil,
                            ReadinessProbe: nil,
                            StartupProbe: nil,
                            Lifecycle: nil,
                            TerminationMessagePath: "/dev/termination-log",
                            TerminationMessagePolicy: "File",
                            ImagePullPolicy: "IfNotPresent",
                            SecurityContext: nil,
                            Stdin: false,
                            StdinOnce: false,
                            TTY: false,
                        },
                    ],
                    EphemeralContainers: nil,
                    RestartPolicy: "Never",
                    TerminationGracePeriodSeconds: 30,
                    ActiveDeadlineSeconds: nil,
                    DNSPolicy: "ClusterFirst",
                    NodeSelector: {
                        "fizz-54d7a2c1-2a70-4c82-a7b4-3fee7394e883": "buzz",
                        "foo-ca6146d2-85d4-4b06-a635-605b2d6f048f": "bar",
                    },
                    ServiceAccountName: "default",
                    DeprecatedServiceAccount: "default",
                    AutomountServiceAccountToken: false,
                    NodeName: "i-03840b5a70dc4c0e3",
                    HostNetwork: false,
                    HostPID: false,
                    HostIPC: false,
                    ShareProcessNamespace: nil,
                    SecurityContext: {
                        SELinuxOptions: nil,
                        WindowsOptions: nil,
                        RunAsUser: nil,
                        RunAsGroup: nil,
                        RunAsNonRoot: nil,
 ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

[FAILED] timed out while waiting for pod runtimeclass-3956/test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6 to be not pending
In [It] at: test/e2e/node/runtimeclass.go:167 @ 01/28/23 07:10:41.453
< Exit [It] should run a Pod requesting a RuntimeClass with scheduling without taints  - test/e2e/node/runtimeclass.go:131 @ 01/28/23 07:10:41.453 (5m21.676s)
> Enter [AfterEach] [sig-node] RuntimeClass - test/e2e/framework/node/init/init.go:33 @ 01/28/23 07:10:41.453
Jan 28 07:10:41.453: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
< Exit [AfterEach] [sig-node] RuntimeClass - test/e2e/framework/node/init/init.go:33 @ 01/28/23 07:10:41.485 (32ms)
> Enter [DeferCleanup (Each)] [sig-node] RuntimeClass - test/e2e/node/runtimeclass.go:152 @ 01/28/23 07:10:41.485
STEP: removing the label fizz-54d7a2c1-2a70-4c82-a7b4-3fee7394e883 off the node i-03840b5a70dc4c0e3 - test/e2e/framework/node/helper.go:72 @ 01/28/23 07:10:41.486
STEP: verifying the node doesn't have the label fizz-54d7a2c1-2a70-4c82-a7b4-3fee7394e883 - test/e2e/framework/node/helper.go:75 @ 01/28/23 07:10:41.556
< Exit [DeferCleanup (Each)] [sig-node] RuntimeClass - test/e2e/node/runtimeclass.go:152 @ 01/28/23 07:10:41.592 (107ms)
> Enter [DeferCleanup (Each)] [sig-node] RuntimeClass - test/e2e/node/runtimeclass.go:152 @ 01/28/23 07:10:41.592
STEP: removing the label foo-ca6146d2-85d4-4b06-a635-605b2d6f048f off the node i-03840b5a70dc4c0e3 - test/e2e/framework/node/helper.go:72 @ 01/28/23 07:10:41.592
STEP: verifying the node doesn't have the label foo-ca6146d2-85d4-4b06-a635-605b2d6f048f - test/e2e/framework/node/helper.go:75 @ 01/28/23 07:10:41.661
< Exit [DeferCleanup (Each)] [sig-node] RuntimeClass - test/e2e/node/runtimeclass.go:152 @ 01/28/23 07:10:41.696 (104ms)
> Enter [DeferCleanup (Each)] [sig-node] RuntimeClass - test/e2e/framework/node/runtimeclass/runtimeclass.go:64 @ 01/28/23 07:10:41.696
STEP: Deleting pod hostexec-i-03840b5a70dc4c0e3-t5zws in namespace runtimeclass-3956 - test/e2e/framework/pod/delete.go:41 @ 01/28/23 07:10:41.696
< Exit [DeferCleanup (Each)] [sig-node] RuntimeClass - test/e2e/framework/node/runtimeclass/runtimeclass.go:64 @ 01/28/23 07:10:41.735 (39ms)
> Enter [DeferCleanup (Each)] [sig-node] RuntimeClass - test/e2e/framework/metrics/init/init.go:35 @ 01/28/23 07:10:41.735
< Exit [DeferCleanup (Each)] [sig-node] RuntimeClass - test/e2e/framework/metrics/init/init.go:35 @ 01/28/23 07:10:41.735 (0s)
> Enter [DeferCleanup (Each)] [sig-node] RuntimeClass - dump namespaces | framework.go:206 @ 01/28/23 07:10:41.735
STEP: dump namespace information after failure - test/e2e/framework/framework.go:284 @ 01/28/23 07:10:41.735
STEP: Collecting events from namespace "runtimeclass-3956". - test/e2e/framework/debug/dump.go:42 @ 01/28/23 07:10:41.735
STEP: Found 14 events. - test/e2e/framework/debug/dump.go:46 @ 01/28/23 07:10:41.767
Jan 28 07:10:41.767: INFO: At 2023-01-28 07:05:19 +0000 UTC - event for hostexec-i-03840b5a70dc4c0e3-t5zws: {default-scheduler } Scheduled: Successfully assigned runtimeclass-3956/hostexec-i-03840b5a70dc4c0e3-t5zws to i-03840b5a70dc4c0e3
Jan 28 07:10:41.767: INFO: At 2023-01-28 07:05:21 +0000 UTC - event for hostexec-i-03840b5a70dc4c0e3-t5zws: {kubelet i-03840b5a70dc4c0e3} Pulling: Pulling image "registry.k8s.io/e2e-test-images/agnhost:2.43"
Jan 28 07:10:41.767: INFO: At 2023-01-28 07:05:25 +0000 UTC - event for hostexec-i-03840b5a70dc4c0e3-t5zws: {kubelet i-03840b5a70dc4c0e3} Pulled: Successfully pulled image "registry.k8s.io/e2e-test-images/agnhost:2.43" in 162.827827ms (3.531010478s including waiting)
Jan 28 07:10:41.767: INFO: At 2023-01-28 07:05:25 +0000 UTC - event for hostexec-i-03840b5a70dc4c0e3-t5zws: {kubelet i-03840b5a70dc4c0e3} Created: Created container agnhost-container
Jan 28 07:10:41.767: INFO: At 2023-01-28 07:05:25 +0000 UTC - event for hostexec-i-03840b5a70dc4c0e3-t5zws: {kubelet i-03840b5a70dc4c0e3} Started: Started container agnhost-container
Jan 28 07:10:41.767: INFO: At 2023-01-28 07:05:29 +0000 UTC - event for without-label: {kubelet i-03840b5a70dc4c0e3} Pulling: Pulling image "registry.k8s.io/pause:3.9"
Jan 28 07:10:41.767: INFO: At 2023-01-28 07:05:29 +0000 UTC - event for without-label: {default-scheduler } Scheduled: Successfully assigned runtimeclass-3956/without-label to i-03840b5a70dc4c0e3
Jan 28 07:10:41.767: INFO: At 2023-01-28 07:05:34 +0000 UTC - event for without-label: {kubelet i-03840b5a70dc4c0e3} Pulled: Successfully pulled image "registry.k8s.io/pause:3.9" in 773.408806ms (4.252438111s including waiting)
Jan 28 07:10:41.767: INFO: At 2023-01-28 07:05:34 +0000 UTC - event for without-label: {kubelet i-03840b5a70dc4c0e3} Created: Created container without-label
Jan 28 07:10:41.767: INFO: At 2023-01-28 07:05:34 +0000 UTC - event for without-label: {kubelet i-03840b5a70dc4c0e3} Started: Started container without-label
Jan 28 07:10:41.767: INFO: At 2023-01-28 07:05:41 +0000 UTC - event for test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6: {default-scheduler } Scheduled: Successfully assigned runtimeclass-3956/test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6 to i-03840b5a70dc4c0e3
Jan 28 07:10:41.767: INFO: At 2023-01-28 07:05:41 +0000 UTC - event for test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6: {kubelet i-03840b5a70dc4c0e3} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to get sandbox runtime: no runtime for "test-handler" is configured
Jan 28 07:10:41.767: INFO: At 2023-01-28 07:05:41 +0000 UTC - event for without-label: {kubelet i-03840b5a70dc4c0e3} Killing: Stopping container without-label
Jan 28 07:10:41.767: INFO: At 2023-01-28 07:10:41 +0000 UTC - event for hostexec-i-03840b5a70dc4c0e3-t5zws: {kubelet i-03840b5a70dc4c0e3} Killing: Stopping container agnhost-container
Jan 28 07:10:41.798: INFO: POD                                                              NODE                 PHASE    GRACE  CONDITIONS
Jan 28 07:10:41.798: INFO: test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6  i-03840b5a70dc4c0e3  Pending         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-01-28 07:05:41 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-01-28 07:05:41 +0000 UTC ContainersNotReady containers with unready status: [test]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-01-28 07:05:41 +0000 UTC ContainersNotReady containers with unready status: [test]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-01-28 07:05:41 +0000 UTC  }]
Jan 28 07:10:41.798: INFO: 
Jan 28 07:10:41.831: INFO: Unable to fetch runtimeclass-3956/test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6/test logs: the server rejected our request for an unknown reason (get pods test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6)
Jan 28 07:10:41.863: INFO: 
Logging node info for node i-03840b5a70dc4c0e3
Jan 28 07:10:41.895: INFO: Node Info: &Node{ObjectMeta:{i-03840b5a70dc4c0e3    07b808d1-06d7-4b70-9ca2-ffe0f64e46ec 17088 0 2023-01-28 07:02:12 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:arm64 beta.kubernetes.io/instance-type:t4g.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:arm64 kubernetes.io/hostname:i-03840b5a70dc4c0e3 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t4g.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-03840b5a70dc4c0e3 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-03840b5a70dc4c0e3"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.2.0/24\"":{}}}} } {kubelet Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-28 07:10:32 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-28 07:10:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.2.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-03840b5a70dc4c0e3,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49756430336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{4027252736 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44780787229 0} {<nil>} 44780787229 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{3922395136 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-28 07:10:33 +0000 UTC,LastTransitionTime:2023-01-28 07:02:12 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-28 07:10:33 +0000 UTC,LastTransitionTime:2023-01-28 07:02:12 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-28 07:10:33 +0000 UTC,LastTransitionTime:2023-01-28 07:02:12 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-28 07:10:33 +0000 UTC,LastTransitionTime:2023-01-28 07:02:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.32.174,},NodeAddress{Type:ExternalIP,Address:3.96.175.228,},NodeAddress{Type:InternalDNS,Address:i-03840b5a70dc4c0e3.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-03840b5a70dc4c0e3.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-175-228.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec21d95d2eb5817b39aed905c39e3faa,SystemUUID:ec21d95d-2eb5-817b-39ae-d905c39e3faa,BootID:c0022840-1246-459b-bca8-c2565d451680,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 22.04.1 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,KubeProxyVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,OperatingSystem:linux,Architecture:arm64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:157636062,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:116764683,},ContainerImage{Names:[registry.k8s.io/kube-proxy-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:63181512,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:49808364,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41528818,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40343601,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:28932904,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:23945474,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:22678632,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:22434180,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:22394501,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18267611,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:8502345,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8223819,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:7685305,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:5025027,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:773367,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:772933,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:268051,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:253553,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-026905a6baf56932d kubernetes.io/csi/ebs.csi.aws.com^vol-0efb154ae272203f3],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-026905a6baf56932d,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-0efb154ae272203f3,DevicePath:,},},Config:nil,},}
Jan 28 07:10:41.895: INFO: 
Logging kubelet events for node i-03840b5a70dc4c0e3
Jan 28 07:10:41.945: INFO: 
Logging pods the kubelet thinks is on node i-03840b5a70dc4c0e3
Jan 28 07:10:41.997: INFO: rand-non-local-s2fkz started at 2023-01-28 07:10:33 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:41.997: INFO: 	Container c ready: false, restart count 0
Jan 28 07:10:41.997: INFO: inline-volume-tester-crvf5 started at 2023-01-28 07:08:44 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:41.997: INFO: 	Container csi-volume-tester ready: true, restart count 0
Jan 28 07:10:41.997: INFO: test-webserver-64efaf7f-4a9c-47a1-8ada-c7dd7480fbc1 started at 2023-01-28 07:06:42 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:41.997: INFO: 	Container test-webserver ready: true, restart count 0
Jan 28 07:10:41.997: INFO: inline-volume-tester-8h728 started at 2023-01-28 07:10:29 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:41.997: INFO: 	Container csi-volume-tester ready: true, restart count 0
Jan 28 07:10:41.997: INFO: rand-non-local-d6kmv started at 2023-01-28 07:10:26 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:41.997: INFO: 	Container c ready: false, restart count 0
Jan 28 07:10:41.997: INFO: test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6 started at 2023-01-28 07:05:41 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:41.997: INFO: 	Container test ready: false, restart count 0
Jan 28 07:10:41.997: INFO: liveness-90202397-a66a-408f-a803-050fa0fcc1a1 started at 2023-01-28 07:06:46 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:41.997: INFO: 	Container agnhost-container ready: true, restart count 0
Jan 28 07:10:41.997: INFO: hostexec-i-03840b5a70dc4c0e3-k8v57 started at 2023-01-28 07:10:19 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:41.997: INFO: 	Container agnhost-container ready: true, restart count 0
Jan 28 07:10:41.997: INFO: netserver-0 started at 2023-01-28 07:10:27 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:41.997: INFO: 	Container webserver ready: true, restart count 0
Jan 28 07:10:41.997: INFO: cilium-55gmk started at 2023-01-28 07:02:13 +0000 UTC (1+1 container statuses recorded)
Jan 28 07:10:41.997: INFO: 	Init container clean-cilium-state ready: true, restart count 0
Jan 28 07:10:41.997: INFO: 	Container cilium-agent ready: true, restart count 0
Jan 28 07:10:41.997: INFO: ebs-csi-node-g8bxd started at 2023-01-28 07:02:13 +0000 UTC (0+3 container statuses recorded)
Jan 28 07:10:41.997: INFO: 	Container ebs-plugin ready: true, restart count 0
Jan 28 07:10:41.997: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:10:41.997: INFO: 	Container node-driver-registrar ready: true, restart count 0
Jan 28 07:10:41.997: INFO: test-rolling-update-with-lb-56d9b65d4c-kr5zb started at 2023-01-28 07:09:51 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:41.997: INFO: 	Container agnhost ready: true, restart count 0
Jan 28 07:10:41.997: INFO: simpletest.rc-xgkk6 started at 2023-01-28 07:10:18 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:41.997: INFO: 	Container nginx ready: true, restart count 0
Jan 28 07:10:42.227: INFO: 
Latency metrics for node i-03840b5a70dc4c0e3
Jan 28 07:10:42.227: INFO: 
Logging node info for node i-05f48d9d08fa62714
Jan 28 07:10:42.259: INFO: Node Info: &Node{ObjectMeta:{i-05f48d9d08fa62714    9e0872b7-c7ff-4d52-8535-3a3f499cab86 16904 0 2023-01-28 07:02:13 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:arm64 beta.kubernetes.io/instance-type:t4g.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a io.kubernetes.storage.mock/node:some-mock-node kubernetes.io/arch:arm64 kubernetes.io/hostname:i-05f48d9d08fa62714 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t4g.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-05f48d9d08fa62714 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-2936":"i-05f48d9d08fa62714","csi-hostpath-volume-1438":"i-05f48d9d08fa62714","csi-mock-csi-mock-volumes-fsgroup-mount-6041":"csi-mock-csi-mock-volumes-fsgroup-mount-6041","ebs.csi.aws.com":"i-05f48d9d08fa62714"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.4.0/24\"":{}}}} } {kubelet Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-28 07:10:21 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-28 07:10:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:io.kubernetes.storage.mock/node":{},"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.4.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-05f48d9d08fa62714,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49756430336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{4027252736 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44780787229 0} {<nil>} 44780787229 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{3922395136 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-28 07:10:34 +0000 UTC,LastTransitionTime:2023-01-28 07:02:13 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-28 07:10:34 +0000 UTC,LastTransitionTime:2023-01-28 07:02:13 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-28 07:10:34 +0000 UTC,LastTransitionTime:2023-01-28 07:02:13 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-28 07:10:34 +0000 UTC,LastTransitionTime:2023-01-28 07:02:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.56.199,},NodeAddress{Type:ExternalIP,Address:3.96.212.120,},NodeAddress{Type:InternalDNS,Address:i-05f48d9d08fa62714.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-05f48d9d08fa62714.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-96-212-120.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec223aa27d20f057bcefa98a58656d96,SystemUUID:ec223aa2-7d20-f057-bcef-a98a58656d96,BootID:4b7e67df-7ab4-44b0-bb7b-1a2eb086bdad,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 22.04.1 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,KubeProxyVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,OperatingSystem:linux,Architecture:arm64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:157636062,},ContainerImage{Names:[registry.k8s.io/kube-proxy-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:63181512,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:49808364,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:47953400,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41528818,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40343601,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:28932904,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:23945474,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:22678632,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:22434180,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:22394501,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18267611,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:13766234,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:8502345,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8223819,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:7685305,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6860089,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:5025027,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:773367,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:772933,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:268051,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:253553,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-1438^d37ebe8d-9eda-11ed-a68a-02af83b5d29e kubernetes.io/csi/ebs.csi.aws.com^vol-04d43a7e481989ce9],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-04d43a7e481989ce9,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-1438^d37ebe8d-9eda-11ed-a68a-02af83b5d29e,DevicePath:,},},Config:nil,},}
Jan 28 07:10:42.259: INFO: 
Logging kubelet events for node i-05f48d9d08fa62714
Jan 28 07:10:42.296: INFO: 
Logging pods the kubelet thinks is on node i-05f48d9d08fa62714
Jan 28 07:10:42.374: INFO: csi-mockplugin-0 started at 2023-01-28 07:10:06 +0000 UTC (0+4 container statuses recorded)
Jan 28 07:10:42.374: INFO: 	Container busybox ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container csi-provisioner ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container driver-registrar ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container mock ready: true, restart count 0
Jan 28 07:10:42.374: INFO: hostpath-injector started at 2023-01-28 07:10:21 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:42.374: INFO: 	Container hostpath-injector ready: true, restart count 0
Jan 28 07:10:42.374: INFO: netserver-1 started at 2023-01-28 07:10:27 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:42.374: INFO: 	Container webserver ready: true, restart count 0
Jan 28 07:10:42.374: INFO: inline-volume-tester-xhljx started at 2023-01-28 07:10:11 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:42.374: INFO: 	Container csi-volume-tester ready: true, restart count 0
Jan 28 07:10:42.374: INFO: hostexec-i-05f48d9d08fa62714-b6bmx started at 2023-01-28 07:10:38 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:42.374: INFO: 	Container agnhost-container ready: true, restart count 0
Jan 28 07:10:42.374: INFO: pvc-volume-tester-cxh5t started at 2023-01-28 07:10:14 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:42.374: INFO: 	Container volume-tester ready: true, restart count 0
Jan 28 07:10:42.374: INFO: csi-hostpathplugin-0 started at 2023-01-28 07:10:14 +0000 UTC (0+7 container statuses recorded)
Jan 28 07:10:42.374: INFO: 	Container csi-attacher ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container csi-provisioner ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container csi-resizer ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container csi-snapshotter ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container hostpath ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container node-driver-registrar ready: true, restart count 0
Jan 28 07:10:42.374: INFO: cilium-wfd8c started at 2023-01-28 07:02:14 +0000 UTC (1+1 container statuses recorded)
Jan 28 07:10:42.374: INFO: 	Init container clean-cilium-state ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container cilium-agent ready: true, restart count 0
Jan 28 07:10:42.374: INFO: ebs-csi-node-9tl4h started at 2023-01-28 07:02:14 +0000 UTC (0+3 container statuses recorded)
Jan 28 07:10:42.374: INFO: 	Container ebs-plugin ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container node-driver-registrar ready: true, restart count 0
Jan 28 07:10:42.374: INFO: csi-hostpathplugin-0 started at 2023-01-28 07:10:11 +0000 UTC (0+7 container statuses recorded)
Jan 28 07:10:42.374: INFO: 	Container csi-attacher ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container csi-provisioner ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container csi-resizer ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container csi-snapshotter ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container hostpath ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:10:42.374: INFO: 	Container node-driver-registrar ready: true, restart count 0
Jan 28 07:10:42.374: INFO: coredns-559769c974-d52jw started at 2023-01-28 07:02:50 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:42.374: INFO: 	Container coredns ready: true, restart count 0
Jan 28 07:10:42.374: INFO: busybox-f3ff15d9-cd6d-4601-9401-47fc83b2fd03 started at 2023-01-28 07:09:35 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:42.374: INFO: 	Container busybox ready: true, restart count 0
Jan 28 07:10:42.374: INFO: test-rolling-update-with-lb-56d9b65d4c-rqzkr started at 2023-01-28 07:09:51 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:42.374: INFO: 	Container agnhost ready: true, restart count 0
Jan 28 07:10:42.717: INFO: 
Latency metrics for node i-05f48d9d08fa62714
Jan 28 07:10:42.717: INFO: 
Logging node info for node i-0ab74bbb764104081
Jan 28 07:10:42.748: INFO: Node Info: &Node{ObjectMeta:{i-0ab74bbb764104081    96aa74ac-06e1-445f-9738-5fa3715ba0be 16873 0 2023-01-28 07:02:11 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:arm64 beta.kubernetes.io/instance-type:t4g.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:arm64 kubernetes.io/hostname:i-0ab74bbb764104081 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t4g.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0ab74bbb764104081 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0ab74bbb764104081"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-01-28 07:02:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.1.0/24\"":{}}}} } {kubelet Update v1 2023-01-28 07:02:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-28 07:02:12 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-28 07:10:28 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-28 07:10:32 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.1.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0ab74bbb764104081,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49756430336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{4027248640 0} {<nil>} 3932860Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44780787229 0} {<nil>} 44780787229 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{3922391040 0} {<nil>} 3830460Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-28 07:10:32 +0000 UTC,LastTransitionTime:2023-01-28 07:02:11 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-28 07:10:32 +0000 UTC,LastTransitionTime:2023-01-28 07:02:11 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-28 07:10:32 +0000 UTC,LastTransitionTime:2023-01-28 07:02:11 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-28 07:10:32 +0000 UTC,LastTransitionTime:2023-01-28 07:02:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.40.17,},NodeAddress{Type:ExternalIP,Address:35.183.243.152,},NodeAddress{Type:InternalDNS,Address:i-0ab74bbb764104081.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0ab74bbb764104081.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-35-183-243-152.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec224fe09fc7d0b0236a5ba8f22f67e1,SystemUUID:ec224fe0-9fc7-d0b0-236a-5ba8f22f67e1,BootID:cf011503-4fe6-4c28-8708-a19a9015457c,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 22.04.1 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,KubeProxyVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,OperatingSystem:linux,Architecture:arm64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:157636062,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:116764683,},ContainerImage{Names:[registry.k8s.io/kube-proxy-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:63181512,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:49808364,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:47953400,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40343601,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:28932904,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:23945474,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:22678632,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:22434180,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:22394501,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:21735644,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:20848413,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:20565480,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:68d396900aeaa072c1f27289485fdac29834045a6f3ffe369bf389d830ef572d registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.6],SizeBytes:19007623,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18267611,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:017727efcfeb7d053af68e51436ce8e65edbc6ca573720afb4f79c8594036955 registry.k8s.io/coredns/coredns:v1.10.0],SizeBytes:13766234,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:8502345,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8223819,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:7685305,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:6883347,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6860089,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2003505,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:773367,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:772933,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:268051,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:253553,},},VolumesInUse:[kubernetes.io/csi/ebs.csi.aws.com^vol-000e35b5b5a58246b kubernetes.io/csi/ebs.csi.aws.com^vol-069e3cf383f8f2125],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-000e35b5b5a58246b,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/ebs.csi.aws.com^vol-069e3cf383f8f2125,DevicePath:,},},Config:nil,},}
Jan 28 07:10:42.748: INFO: 
Logging kubelet events for node i-0ab74bbb764104081
Jan 28 07:10:42.783: INFO: 
Logging pods the kubelet thinks is on node i-0ab74bbb764104081
Jan 28 07:10:42.821: INFO: inline-volume-tester-fbgl8 started at 2023-01-28 07:10:07 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:42.821: INFO: 	Container csi-volume-tester ready: true, restart count 0
Jan 28 07:10:42.821: INFO: cilium-lfmv7 started at 2023-01-28 07:02:12 +0000 UTC (1+1 container statuses recorded)
Jan 28 07:10:42.821: INFO: 	Init container clean-cilium-state ready: true, restart count 0
Jan 28 07:10:42.821: INFO: 	Container cilium-agent ready: true, restart count 0
Jan 28 07:10:42.821: INFO: ebs-csi-controller-555658678f-dfmf4 started at 2023-01-28 07:02:22 +0000 UTC (0+5 container statuses recorded)
Jan 28 07:10:42.821: INFO: 	Container csi-attacher ready: true, restart count 0
Jan 28 07:10:42.821: INFO: 	Container csi-provisioner ready: true, restart count 0
Jan 28 07:10:42.821: INFO: 	Container csi-resizer ready: true, restart count 0
Jan 28 07:10:42.821: INFO: 	Container ebs-plugin ready: true, restart count 0
Jan 28 07:10:42.821: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:10:42.821: INFO: ebs-csi-node-zkczw started at 2023-01-28 07:02:12 +0000 UTC (0+3 container statuses recorded)
Jan 28 07:10:42.821: INFO: 	Container ebs-plugin ready: true, restart count 0
Jan 28 07:10:42.821: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:10:42.821: INFO: 	Container node-driver-registrar ready: true, restart count 0
Jan 28 07:10:42.821: INFO: coredns-559769c974-fnksb started at 2023-01-28 07:02:22 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:42.821: INFO: 	Container coredns ready: true, restart count 0
Jan 28 07:10:42.821: INFO: webhook-to-be-mutated started at 2023-01-28 07:10:06 +0000 UTC (1+1 container statuses recorded)
Jan 28 07:10:42.821: INFO: 	Init container webhook-added-init-container ready: false, restart count 0
Jan 28 07:10:42.821: INFO: 	Container example ready: false, restart count 0
Jan 28 07:10:42.821: INFO: simpletest.rc-q9tth started at 2023-01-28 07:10:18 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:42.821: INFO: 	Container nginx ready: true, restart count 0
Jan 28 07:10:42.821: INFO: hostexec-i-0ab74bbb764104081-cpv5c started at 2023-01-28 07:10:22 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:42.821: INFO: 	Container agnhost-container ready: true, restart count 0
Jan 28 07:10:42.821: INFO: coredns-autoscaler-7cb5c5b969-lhx5j started at 2023-01-28 07:02:22 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:42.821: INFO: 	Container autoscaler ready: true, restart count 0
Jan 28 07:10:42.821: INFO: inline-volume-tester2-pmhf7 started at 2023-01-28 07:10:25 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:42.821: INFO: 	Container csi-volume-tester ready: true, restart count 0
Jan 28 07:10:42.821: INFO: netserver-2 started at 2023-01-28 07:10:27 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:42.821: INFO: 	Container webserver ready: false, restart count 0
Jan 28 07:10:43.014: INFO: 
Latency metrics for node i-0ab74bbb764104081
Jan 28 07:10:43.014: INFO: 
Logging node info for node i-0c0709f8c7952ec00
Jan 28 07:10:43.047: INFO: Node Info: &Node{ObjectMeta:{i-0c0709f8c7952ec00    ee8d889d-42fa-49b7-9c7f-f8b5d1a036f5 9357 0 2023-01-28 06:59:47 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:arm64 beta.kubernetes.io/instance-type:c6g.large beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kops.k8s.io/kops-controller-pki: kubernetes.io/arch:arm64 kubernetes.io/hostname:i-0c0709f8c7952ec00 kubernetes.io/os:linux node-role.kubernetes.io/control-plane: node.kubernetes.io/exclude-from-external-load-balancers: node.kubernetes.io/instance-type:c6g.large topology.ebs.csi.aws.com/zone:ca-central-1a topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[csi.volume.kubernetes.io/nodeid:{"ebs.csi.aws.com":"i-0c0709f8c7952ec00"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-01-28 06:59:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {protokube Update v1 2023-01-28 07:00:14 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kops.k8s.io/kops-controller-pki":{},"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}}} } {aws-cloud-controller-manager Update v1 2023-01-28 07:01:06 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-28 07:01:06 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-01-28 07:01:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-01-28 07:07:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.0.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0c0709f8c7952ec00,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/control-plane,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[100.96.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49756430336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{4002131968 0} {<nil>} 3908332Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44780787229 0} {<nil>} 44780787229 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{3897274368 0} {<nil>} 3805932Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-28 07:07:36 +0000 UTC,LastTransitionTime:2023-01-28 06:59:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-28 07:07:36 +0000 UTC,LastTransitionTime:2023-01-28 06:59:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-28 07:07:36 +0000 UTC,LastTransitionTime:2023-01-28 06:59:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-28 07:07:36 +0000 UTC,LastTransitionTime:2023-01-28 07:01:08 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.56.68,},NodeAddress{Type:ExternalIP,Address:99.79.32.119,},NodeAddress{Type:InternalDNS,Address:i-0c0709f8c7952ec00.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0c0709f8c7952ec00.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-99-79-32-119.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2a6c102ceaf46a9a6436a04797b985,SystemUUID:ec2a6c10-2cea-f46a-9a64-36a04797b985,BootID:32e68828-ba43-463f-b849-18b0db770fe8,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 22.04.1 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,KubeProxyVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,OperatingSystem:linux,Architecture:arm64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/etcd@sha256:9ce33ba33d8e738a5b85ed50b5080ac746deceed4a7496c550927a7a19ca3b6d registry.k8s.io/etcd:3.5.0-0],SizeBytes:157797353,},ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:157636062,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646 registry.k8s.io/etcd:3.4.3-0],SizeBytes:151677080,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 registry.k8s.io/etcd:3.4.13-0],SizeBytes:134531559,},ContainerImage{Names:[registry.k8s.io/kube-apiserver-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:131348942,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:0d2ab6b15b4fe0b903b9e81192d942fb39c29be61fe341cd16db5d0165b0435c registry.k8s.io/etcd:3.3.17-0],SizeBytes:126979107,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:121190708,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:17da501f5d2a675be46040422a27b7cc21b8a43895ac998b171db1c346f361f7 registry.k8s.io/etcd:3.3.10-0],SizeBytes:81436097,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:6f72b851544986cb0921b53ea655ec04c36131248f16d4ad110cb3ca0c369dc1 registry.k8s.io/etcd:3.5.4-0],SizeBytes:81120118,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 registry.k8s.io/etcd:3.5.3-0],SizeBytes:81114891,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:80539316,},ContainerImage{Names:[registry.k8s.io/kube-proxy-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:63181512,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20 registry.k8s.io/etcd:3.2.24-1],SizeBytes:61500433,},ContainerImage{Names:[registry.k8s.io/etcdadm/etcd-manager@sha256:81677375343460c8b223d5217ed2ee037d2578b470b3260637a735f57e5bdf04 registry.k8s.io/etcdadm/etcd-manager:v3.0.20230119-slim],SizeBytes:60827222,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:64b9ea357325d5db9f8a723dcf503b5a449177b17ac87d69481e126bb724c263 registry.k8s.io/etcd:3.5.1-0],SizeBytes:59553904,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:56048433,},ContainerImage{Names:[registry.k8s.io/kops/kops-controller:1.27.0-alpha.1],SizeBytes:41190270,},ContainerImage{Names:[registry.k8s.io/kops/dns-controller:1.27.0-alpha.1],SizeBytes:39793987,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:28932904,},ContainerImage{Names:[quay.io/cilium/operator@sha256:a6d24a006a6b92967ac90786b49bc1ac26e5477cf028cd1186efcfc2466484db quay.io/cilium/operator:v1.12.5],SizeBytes:24567287,},ContainerImage{Names:[registry.k8s.io/provider-aws/cloud-controller-manager@sha256:fdeb61e3e42ecd9cca868d550ebdb88dd6341d9e91fcfa9a37e227dab2ad22cb registry.k8s.io/provider-aws/cloud-controller-manager:v1.26.0],SizeBytes:18345226,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:8502345,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:7685305,},ContainerImage{Names:[registry.k8s.io/kops/kube-apiserver-healthcheck:1.27.0-alpha.1],SizeBytes:4691073,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:253553,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jan 28 07:10:43.047: INFO: 
Logging kubelet events for node i-0c0709f8c7952ec00
Jan 28 07:10:43.082: INFO: 
Logging pods the kubelet thinks is on node i-0c0709f8c7952ec00
Jan 28 07:10:43.132: INFO: cilium-operator-6f7d7f696f-qd5rh started at 2023-01-28 07:00:50 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:43.132: INFO: 	Container cilium-operator ready: true, restart count 0
Jan 28 07:10:43.132: INFO: dns-controller-85d56fc6db-brmzv started at 2023-01-28 07:00:50 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:43.132: INFO: 	Container dns-controller ready: true, restart count 0
Jan 28 07:10:43.132: INFO: aws-cloud-controller-manager-wzlg2 started at 2023-01-28 07:00:49 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:43.132: INFO: 	Container aws-cloud-controller-manager ready: true, restart count 0
Jan 28 07:10:43.132: INFO: cilium-w9t7g started at 2023-01-28 07:00:49 +0000 UTC (1+1 container statuses recorded)
Jan 28 07:10:43.132: INFO: 	Init container clean-cilium-state ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Container cilium-agent ready: true, restart count 0
Jan 28 07:10:43.132: INFO: kops-controller-sl7df started at 2023-01-28 07:00:49 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:43.132: INFO: 	Container kops-controller ready: true, restart count 0
Jan 28 07:10:43.132: INFO: kube-apiserver-i-0c0709f8c7952ec00 started at 2023-01-28 06:58:21 +0000 UTC (0+2 container statuses recorded)
Jan 28 07:10:43.132: INFO: 	Container healthcheck ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Container kube-apiserver ready: true, restart count 2
Jan 28 07:10:43.132: INFO: kube-controller-manager-i-0c0709f8c7952ec00 started at 2023-01-28 06:58:21 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:43.132: INFO: 	Container kube-controller-manager ready: true, restart count 4
Jan 28 07:10:43.132: INFO: ebs-csi-node-lgtrp started at 2023-01-28 07:00:49 +0000 UTC (0+3 container statuses recorded)
Jan 28 07:10:43.132: INFO: 	Container ebs-plugin ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Container node-driver-registrar ready: true, restart count 0
Jan 28 07:10:43.132: INFO: kube-scheduler-i-0c0709f8c7952ec00 started at 2023-01-28 06:58:21 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:43.132: INFO: 	Container kube-scheduler ready: true, restart count 0
Jan 28 07:10:43.132: INFO: etcd-manager-events-i-0c0709f8c7952ec00 started at 2023-01-28 06:58:21 +0000 UTC (10+1 container statuses recorded)
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Container etcd-manager ready: true, restart count 0
Jan 28 07:10:43.132: INFO: etcd-manager-main-i-0c0709f8c7952ec00 started at 2023-01-28 06:58:21 +0000 UTC (10+1 container statuses recorded)
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-2-24 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-3-10 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-3-17 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-4-13 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-4-3 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-5-0 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-5-1 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-5-3 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-5-4 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Init container init-etcd-3-5-6 ready: true, restart count 0
Jan 28 07:10:43.132: INFO: 	Container etcd-manager ready: true, restart count 0
Jan 28 07:10:43.312: INFO: 
Latency metrics for node i-0c0709f8c7952ec00
Jan 28 07:10:43.312: INFO: 
Logging node info for node i-0f637d6210a139b70
Jan 28 07:10:43.343: INFO: Node Info: &Node{ObjectMeta:{i-0f637d6210a139b70    e6187f2f-df4a-409d-84ab-b2e2ad79b8cf 16934 0 2023-01-28 07:02:13 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:arm64 beta.kubernetes.io/instance-type:t4g.medium beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ca-central-1 failure-domain.beta.kubernetes.io/zone:ca-central-1a kubernetes.io/arch:arm64 kubernetes.io/hostname:i-0f637d6210a139b70 kubernetes.io/os:linux node-role.kubernetes.io/node: node.kubernetes.io/instance-type:t4g.medium topology.ebs.csi.aws.com/zone:ca-central-1a topology.hostpath.csi/node:i-0f637d6210a139b70 topology.kubernetes.io/region:ca-central-1 topology.kubernetes.io/zone:ca-central-1a] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-attach-9745":"i-0f637d6210a139b70","csi-mock-csi-mock-volumes-capacity-6328":"i-0f637d6210a139b70","ebs.csi.aws.com":"i-0f637d6210a139b70"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{aws-cloud-controller-manager Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {aws-cloud-controller-manager Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:status":{"f:addresses":{"k:{\"type\":\"ExternalDNS\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"ExternalIP\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"Hostname\"}":{"f:address":{}},"k:{\"type\":\"InternalDNS\"}":{".":{},"f:address":{},"f:type":{}}}}} status} {kops-controller Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:node-role.kubernetes.io/node":{}}}} } {kube-controller-manager Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"100.96.3.0/24\"":{}}}} } {kubelet Update v1 2023-01-28 07:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}} } {kube-controller-manager Update v1 2023-01-28 07:10:35 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2023-01-28 07:10:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.ebs.csi.aws.com/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:100.96.3.0/24,DoNotUseExternalID:,ProviderID:aws:///ca-central-1a/i-0f637d6210a139b70,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[100.96.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{49756430336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{4027248640 0} {<nil>} 3932860Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{44780787229 0} {<nil>} 44780787229 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-32Mi: {{0 0} {<nil>} 0 DecimalSI},hugepages-64Ki: {{0 0} {<nil>} 0 DecimalSI},memory: {{3922391040 0} {<nil>} 3830460Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-01-28 07:10:35 +0000 UTC,LastTransitionTime:2023-01-28 07:02:13 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-01-28 07:10:35 +0000 UTC,LastTransitionTime:2023-01-28 07:02:13 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-01-28 07:10:35 +0000 UTC,LastTransitionTime:2023-01-28 07:02:13 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-01-28 07:10:35 +0000 UTC,LastTransitionTime:2023-01-28 07:02:23 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.20.58.154,},NodeAddress{Type:ExternalIP,Address:3.99.165.197,},NodeAddress{Type:InternalDNS,Address:i-0f637d6210a139b70.ca-central-1.compute.internal,},NodeAddress{Type:Hostname,Address:i-0f637d6210a139b70.ca-central-1.compute.internal,},NodeAddress{Type:ExternalDNS,Address:ec2-3-99-165-197.ca-central-1.compute.amazonaws.com,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ec2dcb89700752b2acd2ab3f959203ae,SystemUUID:ec2dcb89-7007-52b2-acd2-ab3f959203ae,BootID:8643c9bc-2d46-4dd1-b176-7c7d8439718a,KernelVersion:5.15.0-1028-aws,OSImage:Ubuntu 22.04.1 LTS,ContainerRuntimeVersion:containerd://1.6.15,KubeletVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,KubeProxyVersion:v1.27.0-alpha.1.69+d7cb1c54a540c9,OperatingSystem:linux,Architecture:arm64,},Images:[]ContainerImage{ContainerImage{Names:[quay.io/cilium/cilium@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5 quay.io/cilium/cilium:v1.12.5],SizeBytes:157636062,},ContainerImage{Names:[registry.k8s.io/kube-proxy-arm64:v1.27.0-alpha.1.69_d7cb1c54a540c9],SizeBytes:63181512,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:49808364,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41528818,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40343601,},ContainerImage{Names:[registry.k8s.io/provider-aws/aws-ebs-csi-driver@sha256:f0c5de192d832e7c1daa6580d4a62e8fa6fc8eabc0917ae4cb7ed4d15e95b59e registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.14.1],SizeBytes:28932904,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:23945474,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:22678632,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:22434180,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:22394501,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:21735644,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:9ebbf9f023e7b41ccee3d52afe39a89e3ddacdbb69269d583abfc25847cfd9e4 registry.k8s.io/sig-storage/csi-resizer:v1.4.0],SizeBytes:20848413,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:8b9c313c05f54fb04f8d430896f5f5904b6cb157df261501b29adc04d2b2dc7b registry.k8s.io/sig-storage/csi-attacher:v3.4.0],SizeBytes:20565480,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18267611,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:8502345,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8223819,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:7685305,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6860089,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:772933,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:268051,},ContainerImage{Names:[registry.k8s.io/pause@sha256:3d380ca8864549e74af4b29c10f9cb0956236dfb01c40ca076fb6c37253234db registry.k8s.io/pause:3.6],SizeBytes:253553,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-attach-9745^d4ffeaae-9eda-11ed-8a90-6ed9d48d9b4b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-attach-9745^d4ffeaae-9eda-11ed-8a90-6ed9d48d9b4b,DevicePath:,},},Config:nil,},}
Jan 28 07:10:43.344: INFO: 
Logging kubelet events for node i-0f637d6210a139b70
Jan 28 07:10:43.379: INFO: 
Logging pods the kubelet thinks is on node i-0f637d6210a139b70
Jan 28 07:10:43.416: INFO: csi-mockplugin-0 started at 2023-01-28 07:10:17 +0000 UTC (0+3 container statuses recorded)
Jan 28 07:10:43.416: INFO: 	Container csi-provisioner ready: true, restart count 0
Jan 28 07:10:43.416: INFO: 	Container driver-registrar ready: true, restart count 0
Jan 28 07:10:43.416: INFO: 	Container mock ready: true, restart count 0
Jan 28 07:10:43.416: INFO: pod-subpath-test-configmap-g64c started at 2023-01-28 07:10:17 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:43.416: INFO: 	Container test-container-subpath-configmap-g64c ready: false, restart count 0
Jan 28 07:10:43.416: INFO: netserver-3 started at 2023-01-28 07:10:27 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:43.416: INFO: 	Container webserver ready: false, restart count 0
Jan 28 07:10:43.416: INFO: csi-mockplugin-attacher-0 started at 2023-01-28 07:10:18 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:43.416: INFO: 	Container csi-attacher ready: true, restart count 0
Jan 28 07:10:43.416: INFO: csi-mockplugin-0 started at 2023-01-28 07:10:29 +0000 UTC (0+3 container statuses recorded)
Jan 28 07:10:43.416: INFO: 	Container csi-provisioner ready: true, restart count 0
Jan 28 07:10:43.416: INFO: 	Container driver-registrar ready: true, restart count 0
Jan 28 07:10:43.416: INFO: 	Container mock ready: true, restart count 0
Jan 28 07:10:43.416: INFO: cilium-lmxnd started at 2023-01-28 07:02:14 +0000 UTC (1+1 container statuses recorded)
Jan 28 07:10:43.416: INFO: 	Init container clean-cilium-state ready: true, restart count 0
Jan 28 07:10:43.416: INFO: 	Container cilium-agent ready: true, restart count 0
Jan 28 07:10:43.416: INFO: ebs-csi-node-28h7q started at 2023-01-28 07:02:14 +0000 UTC (0+3 container statuses recorded)
Jan 28 07:10:43.416: INFO: 	Container ebs-plugin ready: true, restart count 0
Jan 28 07:10:43.416: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:10:43.416: INFO: 	Container node-driver-registrar ready: true, restart count 0
Jan 28 07:10:43.416: INFO: ebs-csi-controller-555658678f-vn4s8 started at 2023-01-28 07:02:24 +0000 UTC (0+5 container statuses recorded)
Jan 28 07:10:43.416: INFO: 	Container csi-attacher ready: true, restart count 0
Jan 28 07:10:43.416: INFO: 	Container csi-provisioner ready: true, restart count 0
Jan 28 07:10:43.416: INFO: 	Container csi-resizer ready: true, restart count 0
Jan 28 07:10:43.416: INFO: 	Container ebs-plugin ready: true, restart count 0
Jan 28 07:10:43.416: INFO: 	Container liveness-probe ready: true, restart count 0
Jan 28 07:10:43.416: INFO: test-rolling-update-with-lb-56d9b65d4c-26cj2 started at 2023-01-28 07:09:51 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:43.416: INFO: 	Container agnhost ready: true, restart count 0
Jan 28 07:10:43.416: INFO: hostexec-i-0f637d6210a139b70-jwjqq started at 2023-01-28 07:10:36 +0000 UTC (0+1 container statuses recorded)
Jan 28 07:10:43.416: INFO: 	Container agnhost-container ready: true, restart count 0
Jan 28 07:10:43.605: INFO: 
Latency metrics for node i-0f637d6210a139b70
END STEP: dump namespace information after failure - test/e2e/framework/framework.go:284 @ 01/28/23 07:10:43.605 (1.869s)
< Exit [DeferCleanup (Each)] [sig-node] RuntimeClass - dump namespaces | framework.go:206 @ 01/28/23 07:10:43.605 (1.87s)
> Enter [DeferCleanup (Each)] [sig-node] RuntimeClass - tear down framework | framework.go:203 @ 01/28/23 07:10:43.605
STEP: Destroying namespace "runtimeclass-3956" for this suite. - test/e2e/framework/framework.go:347 @ 01/28/23 07:10:43.605
< Exit [DeferCleanup (Each)] [sig-node] RuntimeClass - tear down framework | framework.go:203 @ 01/28/23 07:10:43.644 (40ms)
> Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 01/28/23 07:10:43.645
< Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 01/28/23 07:10:43.645 (0s)

				Click to see stdout/stderrfrom junit_01.xml

Find runtimeclass-3956/test-runtimeclass-runtimeclass-3956-non-conflict-runtimecllpxm6 mentions in log files | View test history on testgrid

kubetest2 Test 21m1s

exit status 255
				from junit_runner.xml

Filter through log files | View test history on testgrid

Show 895 Passed Tests

Show 6516 Skipped Tests

Result	FAILURE
Tests	3 failed / 895 succeeded
Started	2023-01-28 06:55
Elapsed	33m27s
Revision	master