This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 0 failed / 0 succeeded
Started2021-10-16 00:25
Elapsed50m6s
Revisionmaster

No Test Failures!


Error lines from build-log.txt

... skipping 175 lines ...
I1016 00:25:57.897864    4918 dumplogs.go:40] /tmp/kops.q7Kkj7i69 toolbox dump --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /etc/aws-ssh/aws-ssh-private --ssh-user ubuntu
I1016 00:25:57.914940    4926 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I1016 00:25:57.915176    4926 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I1016 00:25:57.915213    4926 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true

Cluster.kops.k8s.io "e2e-89d41a7532-e6156.test-cncf-aws.k8s.io" not found
W1016 00:25:58.410054    4918 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1
I1016 00:25:58.410111    4918 down.go:48] /tmp/kops.q7Kkj7i69 delete cluster --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --yes
I1016 00:25:58.425479    4937 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I1016 00:25:58.425565    4937 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I1016 00:25:58.425572    4937 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true

error reading cluster configuration: Cluster.kops.k8s.io "e2e-89d41a7532-e6156.test-cncf-aws.k8s.io" not found
I1016 00:25:58.904363    4918 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip
2021/10/16 00:25:58 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404
I1016 00:25:58.911708    4918 http.go:37] curl https://ip.jsb.workers.dev
I1016 00:25:58.983421    4918 up.go:144] /tmp/kops.q7Kkj7i69 create cluster --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --cloud aws --kubernetes-version v1.21.0 --ssh-public-key /etc/aws-ssh/aws-ssh-public --override cluster.spec.nodePortAccess=0.0.0.0/0 --yes --networking calico --admin-access 34.136.87.54/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones ap-southeast-1a --master-size c5.large
I1016 00:25:58.998232    4949 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I1016 00:25:58.998302    4949 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I1016 00:25:58.998307    4949 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I1016 00:25:59.041822    4949 create_cluster.go:728] Using SSH public key: /etc/aws-ssh/aws-ssh-public
... skipping 44 lines ...
I1016 00:26:30.340960    4918 up.go:181] /tmp/kops.q7Kkj7i69 validate cluster --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --count 10 --wait 15m0s
I1016 00:26:30.356183    4971 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I1016 00:26:30.356399    4971 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I1016 00:26:30.356452    4971 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
Validating cluster e2e-89d41a7532-e6156.test-cncf-aws.k8s.io

W1016 00:26:31.983355    4971 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:26:42.021585    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:26:52.069591    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:27:02.103116    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:27:12.169473    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:27:22.214457    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:27:32.242510    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:27:42.302548    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:27:52.339448    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:28:02.376061    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:28:12.406019    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:28:22.444364    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:28:32.484062    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:28:42.512231    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:28:52.545685    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:29:02.581186    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:29:12.623354    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:29:22.652563    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:29:32.679967    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:29:42.712380    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:29:52.743442    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W1016 00:30:02.789799    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

... skipping 9 lines ...
Machine	i-0e1df3e58c2ad2c9a					machine "i-0e1df3e58c2ad2c9a" has not yet joined cluster
Pod	kube-system/calico-kube-controllers-6b59cd85f8-k4f4z	system-cluster-critical pod "calico-kube-controllers-6b59cd85f8-k4f4z" is pending
Pod	kube-system/calico-node-z47lm				system-node-critical pod "calico-node-z47lm" is pending
Pod	kube-system/coredns-5dc785954d-7sxlv			system-cluster-critical pod "coredns-5dc785954d-7sxlv" is pending
Pod	kube-system/coredns-autoscaler-84d4cfd89c-t7p4j		system-cluster-critical pod "coredns-autoscaler-84d4cfd89c-t7p4j" is pending

Validation Failed
W1016 00:30:17.469225    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

... skipping 8 lines ...
Machine	i-086f2959ed85f9c0d					machine "i-086f2959ed85f9c0d" has not yet joined cluster
Machine	i-0e1df3e58c2ad2c9a					machine "i-0e1df3e58c2ad2c9a" has not yet joined cluster
Pod	kube-system/calico-kube-controllers-6b59cd85f8-k4f4z	system-cluster-critical pod "calico-kube-controllers-6b59cd85f8-k4f4z" is pending
Pod	kube-system/coredns-5dc785954d-7sxlv			system-cluster-critical pod "coredns-5dc785954d-7sxlv" is pending
Pod	kube-system/coredns-autoscaler-84d4cfd89c-t7p4j		system-cluster-critical pod "coredns-autoscaler-84d4cfd89c-t7p4j" is pending

Validation Failed
W1016 00:30:31.016918    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

... skipping 16 lines ...
Pod	kube-system/coredns-5dc785954d-7sxlv					system-cluster-critical pod "coredns-5dc785954d-7sxlv" is pending
Pod	kube-system/coredns-autoscaler-84d4cfd89c-t7p4j				system-cluster-critical pod "coredns-autoscaler-84d4cfd89c-t7p4j" is pending
Pod	kube-system/kube-proxy-ip-172-20-32-74.ap-southeast-1.compute.internal	system-node-critical pod "kube-proxy-ip-172-20-32-74.ap-southeast-1.compute.internal" is pending
Pod	kube-system/kube-proxy-ip-172-20-34-80.ap-southeast-1.compute.internal	system-node-critical pod "kube-proxy-ip-172-20-34-80.ap-southeast-1.compute.internal" is pending
Pod	kube-system/kube-proxy-ip-172-20-45-87.ap-southeast-1.compute.internal	system-node-critical pod "kube-proxy-ip-172-20-45-87.ap-southeast-1.compute.internal" is pending

Validation Failed
W1016 00:30:44.406063    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

... skipping 16 lines ...
Pod	kube-system/calico-node-l6cd7						system-node-critical pod "calico-node-l6cd7" is pending
Pod	kube-system/calico-node-p8hgm						system-node-critical pod "calico-node-p8hgm" is pending
Pod	kube-system/coredns-5dc785954d-7sxlv					system-cluster-critical pod "coredns-5dc785954d-7sxlv" is pending
Pod	kube-system/coredns-autoscaler-84d4cfd89c-t7p4j				system-cluster-critical pod "coredns-autoscaler-84d4cfd89c-t7p4j" is pending
Pod	kube-system/kube-proxy-ip-172-20-42-16.ap-southeast-1.compute.internal	system-node-critical pod "kube-proxy-ip-172-20-42-16.ap-southeast-1.compute.internal" is pending

Validation Failed
W1016 00:30:57.725565    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

... skipping 13 lines ...
Pod	kube-system/calico-node-6hsz4			system-node-critical pod "calico-node-6hsz4" is pending
Pod	kube-system/calico-node-l6cd7			system-node-critical pod "calico-node-l6cd7" is pending
Pod	kube-system/calico-node-p8hgm			system-node-critical pod "calico-node-p8hgm" is pending
Pod	kube-system/coredns-5dc785954d-7sxlv		system-cluster-critical pod "coredns-5dc785954d-7sxlv" is pending
Pod	kube-system/coredns-autoscaler-84d4cfd89c-t7p4j	system-cluster-critical pod "coredns-autoscaler-84d4cfd89c-t7p4j" is pending

Validation Failed
W1016 00:31:10.937788    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

... skipping 11 lines ...
Pod	kube-system/calico-node-6hsz4			system-node-critical pod "calico-node-6hsz4" is not ready (calico-node)
Pod	kube-system/calico-node-l6cd7			system-node-critical pod "calico-node-l6cd7" is not ready (calico-node)
Pod	kube-system/calico-node-p8hgm			system-node-critical pod "calico-node-p8hgm" is not ready (calico-node)
Pod	kube-system/coredns-5dc785954d-7sxlv		system-cluster-critical pod "coredns-5dc785954d-7sxlv" is pending
Pod	kube-system/coredns-autoscaler-84d4cfd89c-t7p4j	system-cluster-critical pod "coredns-autoscaler-84d4cfd89c-t7p4j" is pending

Validation Failed
W1016 00:31:24.363026    4971 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-southeast-1a	Master	c5.large	1	1	ap-southeast-1a
nodes-ap-southeast-1a	Node	t3.medium	4	4	ap-southeast-1a

... skipping 799 lines ...
  	                    	        for cmd in "${commands[@]}"; do
  	                    	...
  	                    	            continue
  	                    	          fi
  	                    	+         if ! validate-hash "${file}" "${hash}"; then
  	                    	-         if [[ -n "${hash}" ]] && ! validate-hash "${file}" "${hash}"; then
  	                    	            echo "== Hash validation of ${url} failed. Retrying. =="
  	                    	            rm -f "${file}"
  	                    	          else
  	                    	-           if [[ -n "${hash}" ]]; then
  	                    	+           echo "== Downloaded ${url} (SHA256 = ${hash}) =="
  	                    	-             echo "== Downloaded ${url} (SHA1 = ${hash}) =="
  	                    	-           else
... skipping 294 lines ...
  	                    	        for cmd in "${commands[@]}"; do
  	                    	...
  	                    	            continue
  	                    	          fi
  	                    	+         if ! validate-hash "${file}" "${hash}"; then
  	                    	-         if [[ -n "${hash}" ]] && ! validate-hash "${file}" "${hash}"; then
  	                    	            echo "== Hash validation of ${url} failed. Retrying. =="
  	                    	            rm -f "${file}"
  	                    	          else
  	                    	-           if [[ -n "${hash}" ]]; then
  	                    	+           echo "== Downloaded ${url} (SHA256 = ${hash}) =="
  	                    	-             echo "== Downloaded ${url} (SHA1 = ${hash}) =="
  	                    	-           else
... skipping 6190 lines ...
evicting pod kube-system/dns-controller-7cf9d66d6d-772jj
I1016 00:36:14.088957    5096 instancegroups.go:658] Waiting for 5s for pods to stabilize after draining.
I1016 00:36:19.089167    5096 instancegroups.go:417] deleting node "ip-172-20-39-45.ap-southeast-1.compute.internal" from kubernetes
I1016 00:36:19.292384    5096 instancegroups.go:591] Stopping instance "i-087b2146c9bdca064", node "ip-172-20-39-45.ap-southeast-1.compute.internal", in group "master-ap-southeast-1a.masters.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io" (this may take a while).
I1016 00:36:19.613907    5096 instancegroups.go:435] waiting for 15s after terminating instance
I1016 00:36:34.614150    5096 instancegroups.go:470] Validating the cluster.
I1016 00:37:04.648555    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:38:04.682356    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:39:04.718346    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:40:04.749729    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:41:04.783508    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:42:04.812480    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:43:04.852151    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:44:04.905183    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:45:04.939828    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:46:04.970862    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:47:05.002822    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:48:05.068200    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:49:05.116066    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:50:05.146770    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:51:05.210953    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:52:05.248506    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:53:05.294603    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:54:05.327605    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:55:05.359235    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:56:05.404789    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:57:05.435269    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:58:05.470871    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 00:59:05.504836    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 01:00:05.549726    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 01:01:05.584405    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 01:02:05.645621    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 01:03:05.694750    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 01:04:05.725187    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 01:05:05.765026    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 01:06:05.795131    5096 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
I1016 01:07:05.828203    5096 instancegroups.go:513] Cluster did not validate within deadline: error listing nodes: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout.
E1016 01:07:05.828247    5096 instancegroups.go:475] Cluster did not validate within 30m0s
Error: master not healthy after update, stopping rolling-update: "error validating cluster after terminating instance: cluster did not validate within a duration of \"30m0s\""
+ kops-finish
+ kubetest2 kops -v=2 --cloud-provider=aws --cluster-name=e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --kops-root=/home/prow/go/src/k8s.io/kops --admin-access= --env=KOPS_FEATURE_FLAGS=SpecOverrideFlag --kops-binary-path=/tmp/kops.kIZypytVK --down
I1016 01:07:05.855152    5116 app.go:59] RunDir for this run: "/logs/artifacts/72592e42-2e17-11ec-8a05-6acfde499713"
I1016 01:07:05.855301    5116 app.go:90] ID for this run: "72592e42-2e17-11ec-8a05-6acfde499713"
I1016 01:07:05.855354    5116 dumplogs.go:40] /tmp/kops.kIZypytVK toolbox dump --name e2e-89d41a7532-e6156.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /etc/aws-ssh/aws-ssh-private --ssh-user ubuntu
I1016 01:07:05.873228    5126 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true
I1016 01:07:05.873419    5126 featureflag.go:166] FeatureFlag "AlphaAllowGCE"=true
I1016 01:07:05.873476    5126 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true
W1016 01:07:49.015961    5126 toolbox_dump.go:168] error listing nodes in cluster: Get "https://api.e2e-89d41a7532-e6156.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 13.250.126.35:443: i/o timeout
2021/10/16 01:07:49 dumping node not registered in kubernetes: 54.254.242.47
2021/10/16 01:07:49 Dumping node 54.254.242.47
2021/10/16 01:07:57 dumping node not registered in kubernetes: 54.255.185.53
2021/10/16 01:07:57 Dumping node 54.255.185.53
2021/10/16 01:08:05 dumping node not registered in kubernetes: 54.151.247.34
2021/10/16 01:08:05 Dumping node 54.151.247.34
... skipping 1487 lines ...