This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 1 failed / 1 succeeded
Started2023-01-17 14:07
Elapsed20m7s
Revisionmaster

Test Failures


kubetest2 Up 15m55s

exit status 1
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 1 Passed Tests

Error lines from build-log.txt

... skipping 135 lines ...
I0117 14:08:53.975678    5507 http.go:37] curl https://storage.googleapis.com/kops-ci/bin/latest-ci-updown-green.txt
I0117 14:08:53.978525    5507 http.go:37] curl https://storage.googleapis.com/kops-ci/bin/1.27.0-alpha.2+v1.27.0-alpha.1-213-g98b8c01dc6/linux/amd64/kops
I0117 14:08:54.858593    5507 local.go:42] ⚙️ ssh-keygen -t ed25519 -N  -q -f /tmp/kops/e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io/id_ed25519
I0117 14:08:54.865447    5507 up.go:44] Cleaning up any leaked resources from previous cluster
I0117 14:08:54.865543    5507 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/3923d490-9670-11ed-824d-f64c9135b4ea/kops toolbox dump --name e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /tmp/kops/e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io/id_ed25519 --ssh-user ec2-user
I0117 14:08:54.865560    5507 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/3923d490-9670-11ed-824d-f64c9135b4ea/kops toolbox dump --name e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /tmp/kops/e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io/id_ed25519 --ssh-user ec2-user
Error: Cluster.kops.k8s.io "e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io" not found
W0117 14:08:55.364054    5507 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1
I0117 14:08:55.364099    5507 down.go:48] /home/prow/go/src/k8s.io/kops/_rundir/3923d490-9670-11ed-824d-f64c9135b4ea/kops delete cluster --name e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io --yes
I0117 14:08:55.364110    5507 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/3923d490-9670-11ed-824d-f64c9135b4ea/kops delete cluster --name e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io --yes
Error: error reading cluster configuration: Cluster.kops.k8s.io "e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io" not found
I0117 14:08:55.916753    5507 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip
2023/01/17 14:08:55 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404
I0117 14:08:55.931605    5507 http.go:37] curl https://ip.jsb.workers.dev
I0117 14:08:56.021313    5507 up.go:167] /home/prow/go/src/k8s.io/kops/_rundir/3923d490-9670-11ed-824d-f64c9135b4ea/kops create cluster --name e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io --cloud aws --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.23.15 --ssh-public-key /tmp/kops/e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io/id_ed25519.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --image=137112412989/amzn2-ami-kernel-5.10-hvm-2.0.20221210.1-x86_64-gp2 --channel=alpha --networking=cilium-eni --container-runtime=containerd --admin-access 34.70.205.183/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones ap-northeast-1a --master-size c5.large
I0117 14:08:56.021353    5507 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/3923d490-9670-11ed-824d-f64c9135b4ea/kops create cluster --name e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io --cloud aws --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.23.15 --ssh-public-key /tmp/kops/e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io/id_ed25519.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --image=137112412989/amzn2-ami-kernel-5.10-hvm-2.0.20221210.1-x86_64-gp2 --channel=alpha --networking=cilium-eni --container-runtime=containerd --admin-access 34.70.205.183/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones ap-northeast-1a --master-size c5.large
I0117 14:08:56.068447    5551 create_cluster.go:882] Using SSH public key: /tmp/kops/e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io/id_ed25519.pub
I0117 14:08:56.583852    5551 new_cluster.go:1338] Cloud Provider ID: "aws"
I0117 14:08:57.957577    5551 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet ap-northeast-1a
... skipping 552 lines ...

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:09:47.876153    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:09:57.913811    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:10:07.950617    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:10:18.003715    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:10:28.047152    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:10:38.082306    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:10:48.131602    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:10:58.167432    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:11:08.206841    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:11:18.242544    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:11:28.283851    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:11:38.325749    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:11:48.365178    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:11:58.418938    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:12:08.458827    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:12:18.506250    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:12:28.544179    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:12:38.596189    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0117 14:12:48.646587    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
W0117 14:13:28.700372    5590 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://api.e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 203.0.113.123:443: i/o timeout
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

NODE STATUS
... skipping 18 lines ...
Pod	kube-system/coredns-autoscaler-557ccb4c66-95n99		system-cluster-critical pod "coredns-autoscaler-557ccb4c66-95n99" is pending
Pod	kube-system/ebs-csi-node-4sw5n				system-node-critical pod "ebs-csi-node-4sw5n" is pending
Pod	kube-system/ebs-csi-node-g8s66				system-node-critical pod "ebs-csi-node-g8s66" is pending
Pod	kube-system/ebs-csi-node-h5kh8				system-node-critical pod "ebs-csi-node-h5kh8" is pending
Pod	kube-system/ebs-csi-node-lfs8c				system-node-critical pod "ebs-csi-node-lfs8c" is pending

Validation Failed
W0117 14:13:42.517753    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 13 lines ...
Pod	kube-system/coredns-autoscaler-557ccb4c66-95n99	system-cluster-critical pod "coredns-autoscaler-557ccb4c66-95n99" is pending
Pod	kube-system/ebs-csi-node-4sw5n			system-node-critical pod "ebs-csi-node-4sw5n" is pending
Pod	kube-system/ebs-csi-node-g8s66			system-node-critical pod "ebs-csi-node-g8s66" is pending
Pod	kube-system/ebs-csi-node-h5kh8			system-node-critical pod "ebs-csi-node-h5kh8" is pending
Pod	kube-system/ebs-csi-node-lfs8c			system-node-critical pod "ebs-csi-node-lfs8c" is pending

Validation Failed
W0117 14:13:55.225077    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 7 lines ...

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is pending

Validation Failed
W0117 14:14:07.968629    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:14:20.651209    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:14:33.359839    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:14:46.002960    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:14:58.758443    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:15:11.499708    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:15:24.085532    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:15:36.634851    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:15:49.170586    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:16:01.704895    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:16:14.291653    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:16:26.865135    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:16:39.480361    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:16:52.031310    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:17:04.663313    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:17:17.148777    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:17:29.771871    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:17:42.964118    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:17:55.641589    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:18:08.355002    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:18:20.846427    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:18:33.457594    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:18:46.014846    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:18:58.548695    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:19:11.114081    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:19:23.650204    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:19:36.206051    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:19:48.779457    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:20:01.388516    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:20:13.926528    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:20:26.548094    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:20:39.294487    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:20:51.796247    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:21:04.411865    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:21:16.900850    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 7 lines ...

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:21:29.834649    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:21:43.063469    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:21:55.584847    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:22:08.176303    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 6 lines ...
ip-172-20-53-240.ap-northeast-1.compute.internal	node		True

VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)

Validation Failed
W0117 14:22:20.702012    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:22:33.237344    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:22:45.746720    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:22:58.341432    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:23:10.980891    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:23:23.692237    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:23:36.265802    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:23:48.866896    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:24:01.402564    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:24:13.920467    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:24:26.390553    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME				ROLE		MACHINETYPE	MIN	MAX	SUBNETS
control-plane-ap-northeast-1a	ControlPlane	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a		Node		t3.medium	4	4	ap-northeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME					MESSAGE
Pod	kube-system/coredns-867df8f45c-cxgbz	system-cluster-critical pod "coredns-867df8f45c-cxgbz" is not ready (coredns)
Pod	kube-system/ebs-csi-node-4sw5n		system-node-critical pod "ebs-csi-node-4sw5n" is not ready (ebs-plugin)
Pod	kube-system/ebs-csi-node-h5kh8		system-node-critical pod "ebs-csi-node-h5kh8" is not ready (ebs-plugin)

Validation Failed
W0117 14:24:38.982220    5590 validate_cluster.go:232] (will retry): cluster not yet healthy
Error: validation failed: wait time exceeded during validation
I0117 14:24:48.993015    5507 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/3923d490-9670-11ed-824d-f64c9135b4ea/kops toolbox dump --name e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /tmp/kops/e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io/id_ed25519 --ssh-user ec2-user
I0117 14:24:48.993079    5507 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/3923d490-9670-11ed-824d-f64c9135b4ea/kops toolbox dump --name e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /tmp/kops/e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io/id_ed25519 --ssh-user ec2-user
2023/01/17 14:25:14 Dumping node ip-172-20-35-78.ap-northeast-1.compute.internal
2023/01/17 14:25:23 error dumping node ip-172-20-35-78.ap-northeast-1.compute.internal: error executing command "sysctl -a": Process exited with status 127
2023/01/17 14:25:23 Dumping node ip-172-20-48-78.ap-northeast-1.compute.internal
2023/01/17 14:25:29 error dumping node ip-172-20-48-78.ap-northeast-1.compute.internal: error executing command "sysctl -a": Process exited with status 127
2023/01/17 14:25:29 Dumping node ip-172-20-50-8.ap-northeast-1.compute.internal
2023/01/17 14:25:35 error dumping node ip-172-20-50-8.ap-northeast-1.compute.internal: error executing command "sysctl -a": Process exited with status 127
2023/01/17 14:25:35 Dumping node ip-172-20-52-61.ap-northeast-1.compute.internal
2023/01/17 14:25:41 error dumping node ip-172-20-52-61.ap-northeast-1.compute.internal: error executing command "sysctl -a": Process exited with status 127
2023/01/17 14:25:41 Dumping node ip-172-20-53-240.ap-northeast-1.compute.internal
2023/01/17 14:25:47 error dumping node ip-172-20-53-240.ap-northeast-1.compute.internal: error executing command "sysctl -a": Process exited with status 127
I0117 14:25:47.857039    5507 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/3923d490-9670-11ed-824d-f64c9135b4ea/kops get cluster --name e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io -o yaml
I0117 14:25:47.857086    5507 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/3923d490-9670-11ed-824d-f64c9135b4ea/kops get cluster --name e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io -o yaml
I0117 14:25:48.366605    5507 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/3923d490-9670-11ed-824d-f64c9135b4ea/kops get instancegroups --name e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io -o yaml
I0117 14:25:48.366641    5507 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/3923d490-9670-11ed-824d-f64c9135b4ea/kops get instancegroups --name e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io -o yaml
I0117 14:25:49.134965    5507 dumplogs.go:98] kubectl cluster-info dump --all-namespaces -o yaml --output-directory /logs/artifacts/cluster-info
I0117 14:25:49.135003    5507 local.go:42] ⚙️ kubectl cluster-info dump --all-namespaces -o yaml --output-directory /logs/artifacts/cluster-info
... skipping 239 lines ...
route-table:rtb-0dd7f7091d558c2c2	ok
vpc:vpc-0f2bebe262e197f60	ok
dhcp-options:dopt-0dadd347594d274fc	ok
Deleted kubectl config for e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io

Deleted cluster: "e2e-e2e-kops-grid-cilium-eni-amzn2-k23.test-cncf-aws.k8s.io"
Error: exit status 1
+ EXIT_VALUE=1
+ set +o xtrace