This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 0 failed / 0 succeeded
Started2021-10-16 22:25
Elapsed46m15s
Revisionmaster

No Test Failures!


Error lines from build-log.txt

... skipping 176 lines ...
I1016 22:25:58.826167    4956 dumplogs.go:40] /tmp/kops.mkYb7AJ7Q toolbox dump --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /etc/aws-ssh/aws-ssh-private --ssh-user ubuntu
I1016 22:25:58.846252    4967 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I1016 22:25:58.846557    4967 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I1016 22:25:58.846568    4967 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true

Cluster.kops.k8s.io "e2e-89d41a7532-e6156.test-cncf-aws.k8s.io" not found
W1016 22:25:59.346904    4956 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1
I1016 22:25:59.346996    4956 down.go:48] /tmp/kops.mkYb7AJ7Q delete cluster --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --yes
I1016 22:25:59.361393    4977 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I1016 22:25:59.361469    4977 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I1016 22:25:59.361474    4977 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true

error reading cluster configuration: Cluster.kops.k8s.io "e2e-89d41a7532-e6156.test-cncf-aws.k8s.io" not found
I1016 22:25:59.910834    4956 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip
2021/10/16 22:25:59 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404
I1016 22:25:59.920253    4956 http.go:37] curl https://ip.jsb.workers.dev
I1016 22:26:00.006966    4956 up.go:144] /tmp/kops.mkYb7AJ7Q create cluster --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --cloud aws --kubernetes-version v1.21.0 --ssh-public-key /etc/aws-ssh/aws-ssh-public --override cluster.spec.nodePortAccess=0.0.0.0/0 --yes --networking calico --admin-access 35.202.104.103/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-east-1a --master-size c5.large
I1016 22:26:00.022824    4987 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I1016 22:26:00.023621    4987 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I1016 22:26:00.023646    4987 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I1016 22:26:00.065648    4987 create_cluster.go:728] Using SSH public key: /etc/aws-ssh/aws-ssh-public
... skipping 44 lines ...
I1016 22:26:24.948969    4956 up.go:181] /tmp/kops.mkYb7AJ7Q validate cluster --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --count 10 --wait 15m0s
I1016 22:26:24.964673    5008 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I1016 22:26:24.964749    5008 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I1016 22:26:24.964756    5008 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
Validating cluster e2e-89d41a7532-e6156.test-cncf-aws.k8s.io

W1016 22:26:25.990694    5008 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:26:36.033896    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:26:46.064305    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:26:56.100137    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
W1016 22:27:06.129446    5008 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:27:16.157577    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:27:26.189467    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:27:36.218560    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:27:46.252405    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:27:56.318346    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:28:06.346162    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:28:16.382505    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:28:26.426949    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:28:36.466424    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:28:46.500307    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:28:56.534620    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:29:06.568292    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:29:16.599414    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:29:26.637688    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:29:36.665755    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:29:46.694186    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 22:29:56.722315    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

... skipping 7 lines ...
Machine	i-0165a06f860fb3ea4				machine "i-0165a06f860fb3ea4" has not yet joined cluster
Machine	i-07729ef9fb82ed8cc				machine "i-07729ef9fb82ed8cc" has not yet joined cluster
Machine	i-0d01fb6402e467e04				machine "i-0d01fb6402e467e04" has not yet joined cluster
Pod	kube-system/coredns-5dc785954d-vf4cd		system-cluster-critical pod "coredns-5dc785954d-vf4cd" is pending
Pod	kube-system/coredns-autoscaler-84d4cfd89c-nht89	system-cluster-critical pod "coredns-autoscaler-84d4cfd89c-nht89" is pending

Validation Failed
W1016 22:30:08.279242    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

... skipping 10 lines ...
Node	ip-172-20-33-228.ec2.internal				node "ip-172-20-33-228.ec2.internal" of role "node" is not ready
Pod	kube-system/calico-node-nmvqw				system-node-critical pod "calico-node-nmvqw" is pending
Pod	kube-system/coredns-5dc785954d-vf4cd			system-cluster-critical pod "coredns-5dc785954d-vf4cd" is pending
Pod	kube-system/coredns-autoscaler-84d4cfd89c-nht89		system-cluster-critical pod "coredns-autoscaler-84d4cfd89c-nht89" is pending
Pod	kube-system/kube-proxy-ip-172-20-33-228.ec2.internal	system-node-critical pod "kube-proxy-ip-172-20-33-228.ec2.internal" is pending

Validation Failed
W1016 22:30:19.320613    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

... skipping 18 lines ...
Pod	kube-system/coredns-5dc785954d-vf4cd			system-cluster-critical pod "coredns-5dc785954d-vf4cd" is pending
Pod	kube-system/coredns-autoscaler-84d4cfd89c-nht89		system-cluster-critical pod "coredns-autoscaler-84d4cfd89c-nht89" is pending
Pod	kube-system/kube-proxy-ip-172-20-40-55.ec2.internal	system-node-critical pod "kube-proxy-ip-172-20-40-55.ec2.internal" is pending
Pod	kube-system/kube-proxy-ip-172-20-45-171.ec2.internal	system-node-critical pod "kube-proxy-ip-172-20-45-171.ec2.internal" is pending
Pod	kube-system/kube-proxy-ip-172-20-53-190.ec2.internal	system-node-critical pod "kube-proxy-ip-172-20-53-190.ec2.internal" is pending

Validation Failed
W1016 22:30:30.275241    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

... skipping 13 lines ...
Pod	kube-system/calico-node-6hrcc		system-node-critical pod "calico-node-6hrcc" is not ready (calico-node)
Pod	kube-system/calico-node-vkwq7		system-node-critical pod "calico-node-vkwq7" is pending
Pod	kube-system/calico-node-w6lz9		system-node-critical pod "calico-node-w6lz9" is pending
Pod	kube-system/coredns-5dc785954d-vdsgz	system-cluster-critical pod "coredns-5dc785954d-vdsgz" is pending
Pod	kube-system/coredns-5dc785954d-vf4cd	system-cluster-critical pod "coredns-5dc785954d-vf4cd" is not ready (coredns)

Validation Failed
W1016 22:30:41.250625    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

... skipping 7 lines ...

VALIDATION ERRORS
KIND	NAME				MESSAGE
Pod	kube-system/calico-node-6hrcc	system-node-critical pod "calico-node-6hrcc" is not ready (calico-node)
Pod	kube-system/calico-node-vkwq7	system-node-critical pod "calico-node-vkwq7" is not ready (calico-node)

Validation Failed
W1016 22:30:52.264809    5008 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

... skipping 799 lines ...
  	                    	        for cmd in "${commands[@]}"; do
  	                    	...
  	                    	            continue
  	                    	          fi
  	                    	+         if ! validate-hash "${file}" "${hash}"; then
  	                    	-         if [[ -n "${hash}" ]] && ! validate-hash "${file}" "${hash}"; then
  	                    	            echo "== Hash validation of ${url} failed. Retrying. =="
  	                    	            rm -f "${file}"
  	                    	          else
  	                    	-           if [[ -n "${hash}" ]]; then
  	                    	+           echo "== Downloaded ${url} (SHA256 = ${hash}) =="
  	                    	-             echo "== Downloaded ${url} (SHA1 = ${hash}) =="
  	                    	-           else
... skipping 294 lines ...
  	                    	        for cmd in "${commands[@]}"; do
  	                    	...
  	                    	            continue
  	                    	          fi
  	                    	+         if ! validate-hash "${file}" "${hash}"; then
  	                    	-         if [[ -n "${hash}" ]] && ! validate-hash "${file}" "${hash}"; then
  	                    	            echo "== Hash validation of ${url} failed. Retrying. =="
  	                    	            rm -f "${file}"
  	                    	          else
  	                    	-           if [[ -n "${hash}" ]]; then
  	                    	+           echo "== Downloaded ${url} (SHA256 = ${hash}) =="
  	                    	-             echo "== Downloaded ${url} (SHA1 = ${hash}) =="
  	                    	-           else
... skipping 4382 lines ...
evicting pod kube-system/dns-controller-7cf9d66d6d-k9wsg
I1016 22:34:25.782659    5133 instancegroups.go:658] Waiting for 5s for pods to stabilize after draining.
I1016 22:34:30.786875    5133 instancegroups.go:417] deleting node "ip-172-20-48-211.ec2.internal" from kubernetes
I1016 22:34:30.828983    5133 instancegroups.go:591] Stopping instance "i-030e1d98002d1e348", node "ip-172-20-48-211.ec2.internal", in group "master-us-east-1a.masters.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io" (this may take a while).
I1016 22:34:30.990371    5133 instancegroups.go:435] waiting for 15s after terminating instance
I1016 22:34:45.990652    5133 instancegroups.go:470] Validating the cluster.
I1016 22:35:16.022254    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:36:16.058648    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:37:16.112443    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:38:16.147709    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:39:16.180579    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:40:16.222136    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:41:16.259696    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:42:16.292584    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:43:16.335076    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:44:16.380876    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:45:16.415704    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:46:16.449695    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:47:16.481956    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:48:16.513485    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:49:16.549391    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:50:16.579717    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:51:16.613103    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:52:16.658640    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:53:16.690490    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:54:16.735183    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:55:16.766206    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:56:16.794630    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:57:16.825398    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:58:16.859604    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 22:59:16.922229    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 23:00:16.969588    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 23:01:16.999276    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 23:02:17.035983    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 23:03:17.066549    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 23:04:17.105531    5133 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
I1016 23:05:17.169029    5133 instancegroups.go:513] Cluster did not validate within deadline: error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout.
E1016 23:05:17.169124    5133 instancegroups.go:475] Cluster did not validate within 30m0s
Error: master not healthy after update, stopping rolling-update: "error validating cluster after terminating instance: cluster did not validate within a duration of \"30m0s\""
+ kops-finish
+ kubetest2 kops -v=2 --cloud-provider=aws --cluster-name=e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --kops-root=/home/prow/go/src/k8s.io/kops --admin-access= --env=KOPS_FEATURE_FLAGS=SpecOverrideFlag --kops-binary-path=/tmp/kops.1xmKQbDZD --down
I1016 23:05:17.204112    5152 app.go:59] RunDir for this run: "/logs/artifacts/d9b4506c-2ecf-11ec-8a05-6acfde499713"
I1016 23:05:17.204257    5152 app.go:90] ID for this run: "d9b4506c-2ecf-11ec-8a05-6acfde499713"
I1016 23:05:17.204282    5152 dumplogs.go:40] /tmp/kops.1xmKQbDZD toolbox dump --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /etc/aws-ssh/aws-ssh-private --ssh-user ubuntu
I1016 23:05:17.224840    5163 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true
I1016 23:05:17.224933    5163 featureflag.go:166] FeatureFlag "AlphaAllowGCE"=true
I1016 23:05:17.224940    5163 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true
W1016 23:05:53.702119    5163 toolbox_dump.go:168] error listing nodes in cluster: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 3.93.188.255:443: i/o timeout
2021/10/16 23:05:53 dumping node not registered in kubernetes: 3.85.188.81
2021/10/16 23:05:53 Dumping node 3.85.188.81
2021/10/16 23:05:55 dumping node not registered in kubernetes: 34.235.127.250
2021/10/16 23:05:55 Dumping node 34.235.127.250
2021/10/16 23:05:57 dumping node not registered in kubernetes: 18.212.217.197
2021/10/16 23:05:57 Dumping node 18.212.217.197
... skipping 1479 lines ...