This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 0 failed / 0 succeeded
Started2021-10-17 22:25
Elapsed46m4s
Revisionmaster

No Test Failures!


Error lines from build-log.txt

... skipping 176 lines ...
I1017 22:25:54.842193    4901 dumplogs.go:40] /tmp/kops.vJ07VexZx toolbox dump --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /etc/aws-ssh/aws-ssh-private --ssh-user ubuntu
I1017 22:25:54.866046    4914 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I1017 22:25:54.866934    4914 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I1017 22:25:54.866943    4914 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true

Cluster.kops.k8s.io "e2e-89d41a7532-e6156.test-cncf-aws.k8s.io" not found
W1017 22:25:55.397867    4901 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1
I1017 22:25:55.397907    4901 down.go:48] /tmp/kops.vJ07VexZx delete cluster --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --yes
I1017 22:25:55.410223    4924 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I1017 22:25:55.410287    4924 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I1017 22:25:55.410291    4924 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true

error reading cluster configuration: Cluster.kops.k8s.io "e2e-89d41a7532-e6156.test-cncf-aws.k8s.io" not found
I1017 22:25:55.905358    4901 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip
2021/10/17 22:25:55 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404
I1017 22:25:55.913185    4901 http.go:37] curl https://ip.jsb.workers.dev
I1017 22:25:55.985435    4901 up.go:144] /tmp/kops.vJ07VexZx create cluster --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --cloud aws --kubernetes-version v1.21.0 --ssh-public-key /etc/aws-ssh/aws-ssh-public --override cluster.spec.nodePortAccess=0.0.0.0/0 --yes --networking calico --admin-access 35.194.23.235/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-west-1a --master-size c5.large
I1017 22:25:56.001698    4934 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I1017 22:25:56.001771    4934 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I1017 22:25:56.001776    4934 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I1017 22:25:56.045038    4934 create_cluster.go:728] Using SSH public key: /etc/aws-ssh/aws-ssh-public
... skipping 44 lines ...
I1017 22:26:20.645668    4901 up.go:181] /tmp/kops.vJ07VexZx validate cluster --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --count 10 --wait 15m0s
I1017 22:26:20.661012    4954 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I1017 22:26:20.661088    4954 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I1017 22:26:20.661093    4954 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
Validating cluster e2e-89d41a7532-e6156.test-cncf-aws.k8s.io

W1017 22:26:21.716910    4954 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:26:31.746702    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:26:41.774824    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:26:51.808368    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
W1017 22:27:01.836939    4954 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:27:11.867284    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:27:21.895422    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
W1017 22:27:31.935040    4954 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:27:41.969817    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:27:52.002685    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:28:02.049330    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:28:12.078982    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:28:22.109435    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:28:32.156133    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:28:42.201069    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:28:52.243217    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:29:02.293332    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:29:12.329150    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:29:22.359286    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:29:32.391354    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1017 22:29:42.427906    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
W1017 22:30:22.466206    4954 validate_cluster.go:173] (will retry): unexpected error during validation: error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 203.0.113.123:443: i/o timeout
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

NODE STATUS
... skipping 13 lines ...
Pod	kube-system/calico-node-nqq4m						system-node-critical pod "calico-node-nqq4m" is pending
Pod	kube-system/calico-node-sl9ss						system-node-critical pod "calico-node-sl9ss" is not ready (calico-node)
Pod	kube-system/coredns-5dc785954d-p9vqc					system-cluster-critical pod "coredns-5dc785954d-p9vqc" is pending
Pod	kube-system/coredns-autoscaler-84d4cfd89c-d47qt				system-cluster-critical pod "coredns-autoscaler-84d4cfd89c-d47qt" is pending
Pod	kube-system/kube-proxy-ip-172-20-53-76.us-west-1.compute.internal	system-node-critical pod "kube-proxy-ip-172-20-53-76.us-west-1.compute.internal" is pending

Validation Failed
W1017 22:30:34.075298    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

... skipping 11 lines ...
Pod	kube-system/calico-node-8vgjg			system-node-critical pod "calico-node-8vgjg" is pending
Pod	kube-system/calico-node-dpw77			system-node-critical pod "calico-node-dpw77" is not ready (calico-node)
Pod	kube-system/calico-node-nqq4m			system-node-critical pod "calico-node-nqq4m" is not ready (calico-node)
Pod	kube-system/calico-node-sl9ss			system-node-critical pod "calico-node-sl9ss" is not ready (calico-node)
Pod	kube-system/coredns-5dc785954d-p9vqc		system-cluster-critical pod "coredns-5dc785954d-p9vqc" is pending

Validation Failed
W1017 22:30:45.266052    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

... skipping 7 lines ...

VALIDATION ERRORS
KIND	NAME				MESSAGE
Pod	kube-system/calico-node-8vgjg	system-node-critical pod "calico-node-8vgjg" is not ready (calico-node)
Pod	kube-system/calico-node-nqq4m	system-node-critical pod "calico-node-nqq4m" is not ready (calico-node)

Validation Failed
W1017 22:30:56.463931    4954 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-west-1a	Master	c5.large	1	1	us-west-1a
nodes-us-west-1a	Node	t3.medium	4	4	us-west-1a

... skipping 806 lines ...
  	                    	        for cmd in "${commands[@]}"; do
  	                    	...
  	                    	            continue
  	                    	          fi
  	                    	+         if ! validate-hash "${file}" "${hash}"; then
  	                    	-         if [[ -n "${hash}" ]] && ! validate-hash "${file}" "${hash}"; then
  	                    	            echo "== Hash validation of ${url} failed. Retrying. =="
  	                    	            rm -f "${file}"
  	                    	          else
  	                    	-           if [[ -n "${hash}" ]]; then
  	                    	+           echo "== Downloaded ${url} (SHA256 = ${hash}) =="
  	                    	-             echo "== Downloaded ${url} (SHA1 = ${hash}) =="
  	                    	-           else
... skipping 294 lines ...
  	                    	        for cmd in "${commands[@]}"; do
  	                    	...
  	                    	            continue
  	                    	          fi
  	                    	+         if ! validate-hash "${file}" "${hash}"; then
  	                    	-         if [[ -n "${hash}" ]] && ! validate-hash "${file}" "${hash}"; then
  	                    	            echo "== Hash validation of ${url} failed. Retrying. =="
  	                    	            rm -f "${file}"
  	                    	          else
  	                    	-           if [[ -n "${hash}" ]]; then
  	                    	+           echo "== Downloaded ${url} (SHA256 = ${hash}) =="
  	                    	-             echo "== Downloaded ${url} (SHA1 = ${hash}) =="
  	                    	-           else
... skipping 4379 lines ...
evicting pod kube-system/dns-controller-7cf9d66d6d-mlfxz
I1017 22:34:26.481956    5079 instancegroups.go:658] Waiting for 5s for pods to stabilize after draining.
I1017 22:34:31.482313    5079 instancegroups.go:417] deleting node "ip-172-20-38-221.us-west-1.compute.internal" from kubernetes
I1017 22:34:31.535484    5079 instancegroups.go:591] Stopping instance "i-05688b9223fff40ac", node "ip-172-20-38-221.us-west-1.compute.internal", in group "master-us-west-1a.masters.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io" (this may take a while).
I1017 22:34:31.701102    5079 instancegroups.go:435] waiting for 15s after terminating instance
I1017 22:34:46.701366    5079 instancegroups.go:470] Validating the cluster.
I1017 22:35:16.724071    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:36:16.767392    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:37:16.814127    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:38:16.845951    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:39:16.879334    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:40:16.908525    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:41:16.943197    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:42:16.979162    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:43:17.023185    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:44:17.053205    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:45:17.086045    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:46:17.122321    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:47:17.151821    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:48:17.184203    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:49:17.230901    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:50:17.262398    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:51:17.293211    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:52:17.328617    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:53:17.358088    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:54:17.401195    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:55:17.432903    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:56:17.463863    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:57:17.494685    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:58:17.528275    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 22:59:17.557671    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 23:00:17.588521    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 23:01:17.620660    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 23:02:17.656839    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 23:03:17.691222    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 23:04:17.722377    5079 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
I1017 23:05:17.754856    5079 instancegroups.go:513] Cluster did not validate within deadline: error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout.
E1017 23:05:17.754910    5079 instancegroups.go:475] Cluster did not validate within 30m0s
Error: master not healthy after update, stopping rolling-update: "error validating cluster after terminating instance: cluster did not validate within a duration of \"30m0s\""
+ kops-finish
+ kubetest2 kops -v=2 --cloud-provider=aws --cluster-name=e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --kops-root=/home/prow/go/src/k8s.io/kops --admin-access= --env=KOPS_FEATURE_FLAGS=SpecOverrideFlag --kops-binary-path=/tmp/kops.4ETaQdMbR --down
I1017 23:05:17.780402    5100 app.go:59] RunDir for this run: "/logs/artifacts/04a545a5-2f99-11ec-8a05-6acfde499713"
I1017 23:05:17.780589    5100 app.go:90] ID for this run: "04a545a5-2f99-11ec-8a05-6acfde499713"
I1017 23:05:17.780641    5100 dumplogs.go:40] /tmp/kops.4ETaQdMbR toolbox dump --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /etc/aws-ssh/aws-ssh-private --ssh-user ubuntu
I1017 23:05:17.800439    5109 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true
I1017 23:05:17.800625    5109 featureflag.go:166] FeatureFlag "AlphaAllowGCE"=true
I1017 23:05:17.800634    5109 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true
W1017 23:05:55.079053    5109 toolbox_dump.go:168] error listing nodes in cluster: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.153.47.212:443: i/o timeout
2021/10/17 23:05:55 dumping node not registered in kubernetes: 54.219.12.93
2021/10/17 23:05:55 Dumping node 54.219.12.93
2021/10/17 23:05:58 dumping node not registered in kubernetes: 13.57.209.102
2021/10/17 23:05:58 Dumping node 13.57.209.102
2021/10/17 23:06:01 dumping node not registered in kubernetes: 13.57.203.240
2021/10/17 23:06:01 Dumping node 13.57.203.240
... skipping 1048 lines ...
I1017 23:06:10.421834    5100 dumplogs.go:72] /tmp/kops.4ETaQdMbR get cluster --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io -o yaml
I1017 23:06:10.959236    5100 dumplogs.go:72] /tmp/kops.4ETaQdMbR get instancegroups --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io -o yaml
I1017 23:06:11.772369    5100 dumplogs.go:91] kubectl cluster-info dump --all-namespaces -o yaml --output-directory /logs/artifacts/cluster-info
I1017 23:06:41.817995    5100 dumplogs.go:114] /tmp/kops.4ETaQdMbR toolbox dump --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --private-key /etc/aws-ssh/aws-ssh-private --ssh-user ubuntu -o yaml
I1017 23:06:48.518752    5100 dumplogs.go:143] ssh -i /etc/aws-ssh/aws-ssh-private -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ubuntu@54.219.12.93 -- kubectl cluster-info dump --all-namespaces -o yaml --output-directory /tmp/cluster-info
Warning: Permanently added '54.219.12.93' (ECDSA) to the list of known hosts.
Error from server (Timeout): the server was unable to return a response in the time allotted, but may still be processing the request (get events)
W1017 23:07:49.883837    5100 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1
I1017 23:07:49.883881    5100 down.go:48] /tmp/kops.4ETaQdMbR delete cluster --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --yes
I1017 23:07:49.900790    5162 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true
I1017 23:07:49.900884    5162 featureflag.go:166] FeatureFlag "AlphaAllowGCE"=true
I1017 23:07:49.900891    5162 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true
TYPE			NAME													ID
autoscaling-config	master-us-west-1a.masters.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io					lt-00763073857b03b60
... skipping 442 lines ...