This job view page is being replaced by Spyglass soon. Check out the new job view.
PRolemarkus: Use helm's kubeclient to apply manifests
ResultFAILURE
Tests 0 failed / 0 succeeded
Started2021-06-29 06:33
Elapsed35m5s
Revision57742d1c85ec401096358d576a285845cc3f930a
Refs 11712

No Test Failures!


Error lines from build-log.txt

... skipping 161 lines ...
I0629 06:34:45.616069    4238 dumplogs.go:38] /tmp/kops.HtjUtPqIt toolbox dump --name e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /etc/aws-ssh/aws-ssh-private --ssh-user ubuntu
I0629 06:34:45.636598    4244 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I0629 06:34:45.637447    4244 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I0629 06:34:45.637455    4244 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true

Cluster.kops.k8s.io "e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io" not found
W0629 06:34:46.137915    4238 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1
I0629 06:34:46.137975    4238 down.go:48] /tmp/kops.HtjUtPqIt delete cluster --name e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io --yes
I0629 06:34:46.157747    4254 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I0629 06:34:46.157919    4254 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I0629 06:34:46.157929    4254 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true

error reading cluster configuration: Cluster.kops.k8s.io "e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io" not found
I0629 06:34:46.679877    4238 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip
2021/06/29 06:34:46 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404
I0629 06:34:46.690080    4238 http.go:37] curl https://ip.jsb.workers.dev
I0629 06:34:46.817578    4238 up.go:144] /tmp/kops.HtjUtPqIt create cluster --name e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io --cloud aws --kubernetes-version 1.21.0 --ssh-public-key /etc/aws-ssh/aws-ssh-public --override cluster.spec.nodePortAccess=0.0.0.0/0 --yes --override=cluster.spec.nodeTerminationHandler.enabled=true --admin-access 34.67.40.150/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones eu-west-1a --master-size c5.large
I0629 06:34:46.835533    4269 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I0629 06:34:46.835661    4269 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I0629 06:34:46.835667    4269 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I0629 06:34:46.887050    4269 create_cluster.go:728] Using SSH public key: /etc/aws-ssh/aws-ssh-public
... skipping 44 lines ...
I0629 06:35:13.546430    4238 up.go:181] /tmp/kops.HtjUtPqIt validate cluster --name e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io --count 10 --wait 15m0s
I0629 06:35:13.561636    4290 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I0629 06:35:13.561735    4290 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I0629 06:35:13.561740    4290 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
Validating cluster e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io

W0629 06:35:14.933580    4290 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:35:24.981038    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:35:35.014524    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:35:45.044524    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:35:55.087469    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:36:05.117820    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:36:15.145947    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:36:25.175226    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:36:35.207926    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:36:45.244311    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:36:55.277476    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:37:05.311466    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:37:15.351186    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:37:25.387268    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:37:35.422737    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:37:45.465191    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:37:55.496230    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:38:05.543993    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:38:15.586432    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:38:25.614320    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:38:35.649282    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:38:45.694368    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:38:55.722983    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:39:05.752282    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:39:15.794637    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:39:25.843631    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:39:35.879776    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:39:45.924272    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:39:55.955567    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0629 06:40:06.000331    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

... skipping 7 lines ...
Machine	i-049dc4dca82e75b57				machine "i-049dc4dca82e75b57" has not yet joined cluster
Machine	i-063b900b2c9f38002				machine "i-063b900b2c9f38002" has not yet joined cluster
Machine	i-0a04f4d80e269645a				machine "i-0a04f4d80e269645a" has not yet joined cluster
Pod	kube-system/coredns-autoscaler-6f594f4c58-m696q	system-cluster-critical pod "coredns-autoscaler-6f594f4c58-m696q" is pending
Pod	kube-system/coredns-f45c4bf76-dml6f		system-cluster-critical pod "coredns-f45c4bf76-dml6f" is pending

Validation Failed
W0629 06:40:19.003055    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

... skipping 11 lines ...
Pod	kube-system/aws-node-termination-handler-p5p47				system-node-critical pod "aws-node-termination-handler-p5p47" is pending
Pod	kube-system/coredns-autoscaler-6f594f4c58-m696q				system-cluster-critical pod "coredns-autoscaler-6f594f4c58-m696q" is pending
Pod	kube-system/coredns-f45c4bf76-dml6f					system-cluster-critical pod "coredns-f45c4bf76-dml6f" is pending
Pod	kube-system/kube-proxy-ip-172-20-41-251.eu-west-1.compute.internal	system-node-critical pod "kube-proxy-ip-172-20-41-251.eu-west-1.compute.internal" is pending
Pod	kube-system/kube-proxy-ip-172-20-63-100.eu-west-1.compute.internal	system-node-critical pod "kube-proxy-ip-172-20-63-100.eu-west-1.compute.internal" is pending

Validation Failed
W0629 06:40:31.102648    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

... skipping 5 lines ...

VALIDATION ERRORS
KIND	NAME			MESSAGE
Machine	i-0453a4b04e33f5bb4	machine "i-0453a4b04e33f5bb4" has not yet joined cluster
Machine	i-063b900b2c9f38002	machine "i-063b900b2c9f38002" has not yet joined cluster

Validation Failed
W0629 06:40:43.045838    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

... skipping 9 lines ...
KIND	NAME									MESSAGE
Node	ip-172-20-45-232.eu-west-1.compute.internal				node "ip-172-20-45-232.eu-west-1.compute.internal" of role "node" is not ready
Node	ip-172-20-53-150.eu-west-1.compute.internal				node "ip-172-20-53-150.eu-west-1.compute.internal" of role "node" is not ready
Pod	kube-system/kube-proxy-ip-172-20-45-232.eu-west-1.compute.internal	system-node-critical pod "kube-proxy-ip-172-20-45-232.eu-west-1.compute.internal" is pending
Pod	kube-system/kube-proxy-ip-172-20-53-150.eu-west-1.compute.internal	system-node-critical pod "kube-proxy-ip-172-20-53-150.eu-west-1.compute.internal" is pending

Validation Failed
W0629 06:40:55.112958    4290 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-eu-west-1a	Master	c5.large	1	1	eu-west-1a
nodes-eu-west-1a	Node	t3.medium	4	4	eu-west-1a

... skipping 911 lines ...
  	                    	        for cmd in "${commands[@]}"; do
  	                    	...
  	                    	            continue
  	                    	          fi
  	                    	+         if ! validate-hash "${file}" "${hash}"; then
  	                    	-         if [[ -n "${hash}" ]] && ! validate-hash "${file}" "${hash}"; then
  	                    	            echo "== Hash validation of ${url} failed. Retrying. =="
  	                    	            rm -f "${file}"
  	                    	          else
  	                    	-           if [[ -n "${hash}" ]]; then
  	                    	+           echo "== Downloaded ${url} (SHA256 = ${hash}) =="
  	                    	-             echo "== Downloaded ${url} (SHA1 = ${hash}) =="
  	                    	-           else
... skipping 220 lines ...
  	                    	        for cmd in "${commands[@]}"; do
  	                    	...
  	                    	            continue
  	                    	          fi
  	                    	+         if ! validate-hash "${file}" "${hash}"; then
  	                    	-         if [[ -n "${hash}" ]] && ! validate-hash "${file}" "${hash}"; then
  	                    	            echo "== Hash validation of ${url} failed. Retrying. =="
  	                    	            rm -f "${file}"
  	                    	          else
  	                    	-           if [[ -n "${hash}" ]]; then
  	                    	+           echo "== Downloaded ${url} (SHA256 = ${hash}) =="
  	                    	-             echo "== Downloaded ${url} (SHA1 = ${hash}) =="
  	                    	-           else
... skipping 546 lines ...
evicting pod kube-system/dns-controller-76c6c8f758-gx297
I0629 06:47:17.194331   11736 instancegroups.go:658] Waiting for 5s for pods to stabilize after draining.
I0629 06:47:22.194837   11736 instancegroups.go:417] deleting node "ip-172-20-44-234.eu-west-1.compute.internal" from kubernetes
I0629 06:47:22.303053   11736 instancegroups.go:591] Stopping instance "i-0865f01dc7cf29833", node "ip-172-20-44-234.eu-west-1.compute.internal", in group "master-eu-west-1a.masters.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io" (this may take a while).
I0629 06:47:22.512923   11736 instancegroups.go:435] waiting for 15s after terminating instance
I0629 06:47:37.514019   11736 instancegroups.go:470] Validating the cluster.
I0629 06:48:07.545197   11736 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
I0629 06:49:07.582530   11736 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
I0629 06:50:07.613805   11736 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
I0629 06:51:07.646371   11736 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
I0629 06:52:07.676657   11736 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
I0629 06:53:07.722375   11736 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
I0629 06:54:07.753112   11736 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
I0629 06:55:07.782983   11736 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
I0629 06:56:07.864587   11736 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
I0629 06:57:07.897240   11736 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
I0629 06:58:07.926641   11736 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
I0629 06:59:07.955630   11736 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
I0629 07:00:07.984611   11736 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
I0629 07:01:08.031739   11736 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
I0629 07:02:08.075994   11736 instancegroups.go:516] Cluster did not validate, will retry in "30s": error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
I0629 07:03:08.122622   11736 instancegroups.go:513] Cluster did not validate within deadline: error listing nodes: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout.
E0629 07:03:08.122668   11736 instancegroups.go:475] Cluster did not validate within 15m0s

master not healthy after update, stopping rolling-update: "error validating cluster after terminating instance: cluster did not validate within a duration of \"15m0s\""
+ kops-finish
+ kubetest2 kops -v=2 --cloud-provider=aws --cluster-name=e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io --kops-root=/home/prow/go/src/github.com/kubernetes/kops --admin-access= --env=KOPS_FEATURE_FLAGS=SpecOverrideFlag --kops-binary-path=.bazelbuild/dist/linux/amd64/kops --down
I0629 07:03:08.190589   11756 app.go:59] RunDir for this run: "/logs/artifacts/e841cf47-d8a3-11eb-9a6d-82aa3ded3f7b"
I0629 07:03:08.191704   11756 app.go:90] ID for this run: "e841cf47-d8a3-11eb-9a6d-82aa3ded3f7b"
I0629 07:03:08.191741   11756 dumplogs.go:38] .bazelbuild/dist/linux/amd64/kops toolbox dump --name e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /etc/aws-ssh/aws-ssh-private --ssh-user ubuntu
I0629 07:03:08.210876   11762 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I0629 07:03:08.211214   11762 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I0629 07:03:08.211301   11762 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
W0629 07:03:48.317815   11762 toolbox_dump.go:180] error listing nodes in cluster: Get "https://api.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 54.78.28.178:443: i/o timeout
2021/06/29 07:03:48 dumping node not registered in kubernetes: 52.214.16.13
2021/06/29 07:03:48 Dumping node 52.214.16.13
2021/06/29 07:03:53 dumping node not registered in kubernetes: 34.245.72.115
2021/06/29 07:03:53 Dumping node 34.245.72.115
2021/06/29 07:03:57 dumping node not registered in kubernetes: 52.18.119.38
2021/06/29 07:03:57 Dumping node 52.18.119.38
... skipping 1004 lines ...
  zone: eu-west-1a
vpc:
  id: vpc-07e2631b95fd62ecc
I0629 07:04:11.621003   11756 dumplogs.go:70] .bazelbuild/dist/linux/amd64/kops get cluster --name e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io -o yaml
I0629 07:04:12.230202   11756 dumplogs.go:70] .bazelbuild/dist/linux/amd64/kops get instancegroups --name e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io -o yaml
I0629 07:04:13.005779   11756 dumplogs.go:89] kubectl cluster-info dump --all-namespaces -o yaml --output-directory /logs/artifacts/cluster-info
W0629 07:04:43.085797   11756 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1
I0629 07:04:43.085896   11756 down.go:48] .bazelbuild/dist/linux/amd64/kops delete cluster --name e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io --yes
I0629 07:04:43.103437   11802 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I0629 07:04:43.103557   11802 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I0629 07:04:43.103565   11802 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
TYPE			NAME													ID
autoscaling-config	master-eu-west-1a.masters.e2e-f9b0e6b3f9-omw33.test-cncf-aws.k8s.io					lt-08e1088160abdab79
... skipping 438 lines ...