This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 0 failed / 1 succeeded
Started2022-07-31 12:43
Elapsed34m11s
Revisionmaster

No Test Failures!


Show 1 Passed Tests

Error lines from build-log.txt

... skipping 190 lines ...
I0731 12:44:57.923627    6188 app.go:65] RunDir for this run: "/home/prow/go/src/k8s.io/kops/_rundir/55fa23da-10ce-11ed-bfc8-0eb9b3896f8e"
I0731 12:44:58.040144    6188 app.go:129] ID for this run: "55fa23da-10ce-11ed-bfc8-0eb9b3896f8e"
I0731 12:44:58.040442    6188 local.go:42] ⚙️ ssh-keygen -t ed25519 -N  -q -f /tmp/kops/e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/id_ed25519
I0731 12:44:58.047547    6188 up.go:44] Cleaning up any leaked resources from previous cluster
I0731 12:44:58.047831    6188 dumplogs.go:45] /tmp/kops.udG90uEGR toolbox dump --name e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /tmp/kops/e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/id_ed25519 --ssh-user ubuntu
I0731 12:44:58.047853    6188 local.go:42] ⚙️ /tmp/kops.udG90uEGR toolbox dump --name e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /tmp/kops/e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/id_ed25519 --ssh-user ubuntu
W0731 12:44:58.624562    6188 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1
I0731 12:44:58.624617    6188 down.go:48] /tmp/kops.udG90uEGR delete cluster --name e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io --yes
I0731 12:44:58.624629    6188 local.go:42] ⚙️ /tmp/kops.udG90uEGR delete cluster --name e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io --yes
I0731 12:44:58.640860    6210 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I0731 12:44:58.640954    6210 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true

error reading cluster configuration: Cluster.kops.k8s.io "e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io" not found
I0731 12:44:59.168109    6188 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip
2022/07/31 12:44:59 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404
I0731 12:44:59.181081    6188 http.go:37] curl https://ip.jsb.workers.dev
I0731 12:44:59.299791    6188 up.go:159] /tmp/kops.udG90uEGR create cluster --name e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io --cloud aws --kubernetes-version 1.21.0 --ssh-public-key /tmp/kops/e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/id_ed25519.pub --override cluster.spec.nodePortAccess=0.0.0.0/0 --override=cluster.spec.nodeTerminationHandler.enabled=true --admin-access 35.232.95.31/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones ap-southeast-1a --master-size c5.large
I0731 12:44:59.299893    6188 local.go:42] ⚙️ /tmp/kops.udG90uEGR create cluster --name e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io --cloud aws --kubernetes-version 1.21.0 --ssh-public-key /tmp/kops/e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/id_ed25519.pub --override cluster.spec.nodePortAccess=0.0.0.0/0 --override=cluster.spec.nodeTerminationHandler.enabled=true --admin-access 35.232.95.31/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones ap-southeast-1a --master-size c5.large
I0731 12:44:59.319542    6221 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I0731 12:44:59.319680    6221 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I0731 12:44:59.373339    6221 create_cluster.go:728] Using SSH public key: /tmp/kops/e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/id_ed25519.pub
... skipping 492 lines ...

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:45:56.001817    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:46:06.042289    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:46:16.095913    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:46:26.144554    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:46:36.181569    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:46:46.235314    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:46:56.268325    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:47:06.307110    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:47:16.356378    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:47:26.390811    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:47:36.428133    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:47:46.462535    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:47:56.502117    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:48:06.536629    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:48:16.581925    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:48:26.616533    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:48:36.665919    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:48:46.708003    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:48:56.740850    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:49:06.775191    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:49:16.808280    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:49:26.847125    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:49:36.890991    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:49:46.943401    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0731 12:49:56.982549    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

... skipping 9 lines ...
Machine	i-0faab5cc08c3876f3							machine "i-0faab5cc08c3876f3" has not yet joined cluster
Pod	kube-system/aws-node-termination-handler-wpt8z				system-node-critical pod "aws-node-termination-handler-wpt8z" is pending
Pod	kube-system/coredns-5dc785954d-gr4hh					system-cluster-critical pod "coredns-5dc785954d-gr4hh" is pending
Pod	kube-system/coredns-autoscaler-84d4cfd89c-dxws6				system-cluster-critical pod "coredns-autoscaler-84d4cfd89c-dxws6" is pending
Pod	kube-system/kube-proxy-ip-172-20-39-218.ap-southeast-1.compute.internal	system-node-critical pod "kube-proxy-ip-172-20-39-218.ap-southeast-1.compute.internal" is pending

Validation Failed
W0731 12:50:12.119410    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

... skipping 10 lines ...
Node	ip-172-20-53-9.ap-southeast-1.compute.internal	node "ip-172-20-53-9.ap-southeast-1.compute.internal" of role "node" is not ready
Pod	kube-system/aws-node-termination-handler-bpbdq	system-node-critical pod "aws-node-termination-handler-bpbdq" is pending
Pod	kube-system/aws-node-termination-handler-wpt8z	system-node-critical pod "aws-node-termination-handler-wpt8z" is pending
Pod	kube-system/coredns-5dc785954d-gr4hh		system-cluster-critical pod "coredns-5dc785954d-gr4hh" is pending
Pod	kube-system/coredns-autoscaler-84d4cfd89c-dxws6	system-cluster-critical pod "coredns-autoscaler-84d4cfd89c-dxws6" is pending

Validation Failed
W0731 12:50:25.980461    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

... skipping 8 lines ...
VALIDATION ERRORS
KIND	NAME									MESSAGE
Pod	kube-system/aws-node-termination-handler-hqpvm				system-node-critical pod "aws-node-termination-handler-hqpvm" is pending
Pod	kube-system/aws-node-termination-handler-j28h9				system-node-critical pod "aws-node-termination-handler-j28h9" is pending
Pod	kube-system/kube-proxy-ip-172-20-58-90.ap-southeast-1.compute.internal	system-node-critical pod "kube-proxy-ip-172-20-58-90.ap-southeast-1.compute.internal" is pending

Validation Failed
W0731 12:50:39.389551    6260 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

... skipping 939 lines ...
  	                    	        for cmd in "${commands[@]}"; do
  	                    	...
  	                    	            continue
  	                    	          fi
  	                    	+         if ! validate-hash "${file}" "${hash}"; then
  	                    	-         if [[ -n "${hash}" ]] && ! validate-hash "${file}" "${hash}"; then
  	                    	            echo "== Hash validation of ${url} failed. Retrying. =="
  	                    	            rm -f "${file}"
  	                    	          else
  	                    	-           if [[ -n "${hash}" ]]; then
  	                    	+           echo "== Downloaded ${url} (SHA256 = ${hash}) =="
  	                    	-             echo "== Downloaded ${url} (SHA1 = ${hash}) =="
  	                    	-           else
... skipping 293 lines ...
  	                    	        for cmd in "${commands[@]}"; do
  	                    	...
  	                    	            continue
  	                    	          fi
  	                    	+         if ! validate-hash "${file}" "${hash}"; then
  	                    	-         if [[ -n "${hash}" ]] && ! validate-hash "${file}" "${hash}"; then
  	                    	            echo "== Hash validation of ${url} failed. Retrying. =="
  	                    	            rm -f "${file}"
  	                    	          else
  	                    	-           if [[ -n "${hash}" ]]; then
  	                    	+           echo "== Downloaded ${url} (SHA256 = ${hash}) =="
  	                    	-             echo "== Downloaded ${url} (SHA1 = ${hash}) =="
  	                    	-           else
... skipping 1152 lines ...
WARNING: ignoring DaemonSet-managed Pods: kube-system/aws-node-termination-handler-mwxt2, kube-system/kops-controller-kwtg4
evicting pod kube-system/dns-controller-7fc66c4dd4-bz8tv
I0731 12:53:43.999121    6355 instancegroups.go:660] Waiting for 5s for pods to stabilize after draining.
I0731 12:53:49.000136    6355 instancegroups.go:591] Stopping instance "i-0b07bad97f472c2b6", node "ip-172-20-56-208.ap-southeast-1.compute.internal", in group "master-ap-southeast-1a.masters.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io" (this may take a while).
I0731 12:53:49.393432    6355 instancegroups.go:436] waiting for 15s after terminating instance
I0731 12:54:04.394277    6355 instancegroups.go:470] Validating the cluster.
I0731 12:54:34.433429    6355 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
I0731 12:55:34.487034    6355 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
I0731 12:56:34.526376    6355 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
I0731 12:57:34.564458    6355 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
I0731 12:58:34.600798    6355 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
I0731 12:59:34.651511    6355 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
I0731 13:00:34.686241    6355 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
I0731 13:01:34.721539    6355 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
I0731 13:02:34.758067    6355 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
I0731 13:03:34.796235    6355 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
I0731 13:04:34.829756    6355 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
I0731 13:05:34.869793    6355 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
I0731 13:06:34.909453    6355 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
I0731 13:07:34.950338    6355 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
I0731 13:08:35.006496    6355 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
I0731 13:09:35.051521    6355 instancegroups.go:513] Cluster did not validate within deadline: error listing nodes: Get "https://api.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.254.202.235:443: i/o timeout.
E0731 13:09:35.051570    6355 instancegroups.go:475] Cluster did not validate within 15m0s
Error: master not healthy after update, stopping rolling-update: "error validating cluster after terminating instance: cluster did not validate within a duration of \"15m0s\""
+ kops-finish
+ kubetest2 kops -v=2 --cloud-provider=aws --cluster-name=e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io --kops-root=/home/prow/go/src/k8s.io/kops --admin-access= --env=KOPS_FEATURE_FLAGS=SpecOverrideFlag --kops-binary-path=/tmp/kops.HqHEOLONQ --down
I0731 13:09:35.080089    6373 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true
I0731 13:09:35.081297    6373 app.go:62] The files in RunDir shall not be part of Artifacts
I0731 13:09:35.081326    6373 app.go:63] pass rundir-in-artifacts flag True for RunDir to be part of Artifacts
I0731 13:09:35.081356    6373 app.go:65] RunDir for this run: "/home/prow/go/src/k8s.io/kops/_rundir/55fa23da-10ce-11ed-bfc8-0eb9b3896f8e"
... skipping 17 lines ...
Warning: Permanently added '52.77.236.53' (ECDSA) to the list of known hosts.
I0731 13:13:11.791945    6373 dumplogs.go:248] ssh -i /tmp/kops/e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/id_ed25519 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ubuntu@52.77.236.53 -- rm -rf /tmp/cluster-info
I0731 13:13:11.792000    6373 local.go:42] ⚙️ ssh -i /tmp/kops/e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io/id_ed25519 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ubuntu@52.77.236.53 -- rm -rf /tmp/cluster-info
Warning: Permanently added '52.77.236.53' (ECDSA) to the list of known hosts.
I0731 13:13:14.530030    6373 dumplogs.go:126] kubectl --request-timeout 5s get csinodes --all-namespaces -o yaml
I0731 13:13:14.530093    6373 local.go:42] ⚙️ kubectl --request-timeout 5s get csinodes --all-namespaces -o yaml
W0731 13:13:19.594120    6373 dumplogs.go:132] Failed to get csinodes: exit status 1
I0731 13:13:19.594316    6373 dumplogs.go:126] kubectl --request-timeout 5s get csidrivers --all-namespaces -o yaml
I0731 13:13:19.594332    6373 local.go:42] ⚙️ kubectl --request-timeout 5s get csidrivers --all-namespaces -o yaml
W0731 13:13:24.656033    6373 dumplogs.go:132] Failed to get csidrivers: exit status 1
I0731 13:13:24.656162    6373 dumplogs.go:126] kubectl --request-timeout 5s get storageclasses --all-namespaces -o yaml
I0731 13:13:24.656179    6373 local.go:42] ⚙️ kubectl --request-timeout 5s get storageclasses --all-namespaces -o yaml
W0731 13:13:29.712644    6373 dumplogs.go:132] Failed to get storageclasses: exit status 1
I0731 13:13:29.712774    6373 dumplogs.go:126] kubectl --request-timeout 5s get persistentvolumes --all-namespaces -o yaml
I0731 13:13:29.712805    6373 local.go:42] ⚙️ kubectl --request-timeout 5s get persistentvolumes --all-namespaces -o yaml
W0731 13:13:34.775041    6373 dumplogs.go:132] Failed to get persistentvolumes: exit status 1
I0731 13:13:34.775210    6373 dumplogs.go:126] kubectl --request-timeout 5s get mutatingwebhookconfigurations --all-namespaces -o yaml
I0731 13:13:34.775226    6373 local.go:42] ⚙️ kubectl --request-timeout 5s get mutatingwebhookconfigurations --all-namespaces -o yaml
W0731 13:13:39.843373    6373 dumplogs.go:132] Failed to get mutatingwebhookconfigurations: exit status 1
I0731 13:13:39.843518    6373 dumplogs.go:126] kubectl --request-timeout 5s get validatingwebhookconfigurations --all-namespaces -o yaml
I0731 13:13:39.843530    6373 local.go:42] ⚙️ kubectl --request-timeout 5s get validatingwebhookconfigurations --all-namespaces -o yaml
W0731 13:13:44.906036    6373 dumplogs.go:132] Failed to get validatingwebhookconfigurations: exit status 1
I0731 13:13:44.906099    6373 local.go:42] ⚙️ kubectl --request-timeout 5s get namespaces --no-headers -o custom-columns=name:.metadata.name
W0731 13:13:49.972678    6373 down.go:34] Dumping cluster logs at the start of Down() failed: failed to get namespaces: exit status 1
I0731 13:13:49.972721    6373 down.go:48] /tmp/kops.HqHEOLONQ delete cluster --name e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io --yes
I0731 13:13:49.972732    6373 local.go:42] ⚙️ /tmp/kops.HqHEOLONQ delete cluster --name e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io --yes
I0731 13:13:50.008179    6502 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true
I0731 13:13:50.008278    6502 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true
TYPE			NAME													ID
autoscaling-config	master-ap-southeast-1a.masters.e2e-c0d41e2af2-13250.test-cncf-aws.k8s.io				lt-0a5fb645a5518d11b
... skipping 498 lines ...