e2e-kops-aws-upgrade-kstable-kolatest-to-kci-kolatest-many-addons #1638138386818535424
Recent runs || View in Spyglass
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 0 failed / 1 succeeded |
| Started | |
| Elapsed | 55m37s |
| Revision | master |
kubetest2 Down
... skipping 205 lines ... I0321 11:21:55.791957 6329 app.go:64] RunDir for this run: "/home/prow/go/src/k8s.io/kops/_rundir/521105f4-c7da-11ed-9e5f-baf334638f51" I0321 11:21:55.845437 6329 app.go:128] ID for this run: "521105f4-c7da-11ed-9e5f-baf334638f51" I0321 11:21:55.845731 6329 local.go:42] ⚙️ ssh-keygen -t ed25519 -N -q -f /tmp/kops/e2e-9b8b75964a-36844.test-cncf-aws.k8s.io/id_ed25519 I0321 11:21:55.858266 6329 dumplogs.go:45] /tmp/kops.0I1XZpc5W toolbox dump --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /tmp/kops/e2e-9b8b75964a-36844.test-cncf-aws.k8s.io/id_ed25519 --ssh-user ubuntu I0321 11:21:55.858341 6329 local.go:42] ⚙️ /tmp/kops.0I1XZpc5W toolbox dump --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /tmp/kops/e2e-9b8b75964a-36844.test-cncf-aws.k8s.io/id_ed25519 --ssh-user ubuntu I0321 11:21:55.893771 6341 featureflag.go:167] Unknown FeatureFlag "SpecOverrideFlag" Error: Cluster.kops.k8s.io "e2e-9b8b75964a-36844.test-cncf-aws.k8s.io" not found W0321 11:21:56.349592 6329 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0321 11:21:56.349638 6329 down.go:48] /tmp/kops.0I1XZpc5W delete cluster --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --yes I0321 11:21:56.349649 6329 local.go:42] ⚙️ /tmp/kops.0I1XZpc5W delete cluster --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --yes I0321 11:21:56.382942 6351 featureflag.go:167] Unknown FeatureFlag "SpecOverrideFlag" Error: error reading cluster configuration: Cluster.kops.k8s.io "e2e-9b8b75964a-36844.test-cncf-aws.k8s.io" not found Error: exit status 1 + echo 'kubetest2 down failed' kubetest2 down failed + [[ l == \v ]] ++ kops-base-from-marker latest ++ [[ latest =~ ^https: ]] ++ [[ latest == \l\a\t\e\s\t ]] ++ curl -fs https://storage.googleapis.com/kops-ci/bin/latest-ci-updown-green.txt + KOPS_BASE_URL=https://storage.googleapis.com/kops-ci/bin/1.27.0-alpha.2+v1.27.0-alpha.1-488-gbccbdaccc6 ... skipping 15 lines ... I0321 11:21:58.198384 6386 app.go:64] RunDir for this run: "/home/prow/go/src/k8s.io/kops/_rundir/521105f4-c7da-11ed-9e5f-baf334638f51" I0321 11:21:58.232378 6386 app.go:128] ID for this run: "521105f4-c7da-11ed-9e5f-baf334638f51" I0321 11:21:58.232479 6386 up.go:44] Cleaning up any leaked resources from previous cluster I0321 11:21:58.232525 6386 dumplogs.go:45] /tmp/kops.x4Or16jWl toolbox dump --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /tmp/kops/e2e-9b8b75964a-36844.test-cncf-aws.k8s.io/id_ed25519 --ssh-user ubuntu I0321 11:21:58.232570 6386 local.go:42] ⚙️ /tmp/kops.x4Or16jWl toolbox dump --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /tmp/kops/e2e-9b8b75964a-36844.test-cncf-aws.k8s.io/id_ed25519 --ssh-user ubuntu I0321 11:21:58.262838 6393 featureflag.go:167] Unknown FeatureFlag "SpecOverrideFlag" Error: Cluster.kops.k8s.io "e2e-9b8b75964a-36844.test-cncf-aws.k8s.io" not found W0321 11:21:58.731384 6386 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0321 11:21:58.731428 6386 down.go:48] /tmp/kops.x4Or16jWl delete cluster --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --yes I0321 11:21:58.731440 6386 local.go:42] ⚙️ /tmp/kops.x4Or16jWl delete cluster --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --yes I0321 11:21:58.761772 6404 featureflag.go:167] Unknown FeatureFlag "SpecOverrideFlag" Error: error reading cluster configuration: Cluster.kops.k8s.io "e2e-9b8b75964a-36844.test-cncf-aws.k8s.io" not found I0321 11:21:59.181110 6386 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2023/03/21 11:21:59 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I0321 11:21:59.194734 6386 http.go:37] curl https://ip.jsb.workers.dev I0321 11:21:59.298848 6386 template.go:58] /tmp/kops.x4Or16jWl toolbox template --template tests/e2e/templates/many-addons.yaml.tmpl --output /tmp/kops-template422928388/manifest.yaml --values /tmp/kops-template422928388/values.yaml --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io I0321 11:21:59.298892 6386 local.go:42] ⚙️ /tmp/kops.x4Or16jWl toolbox template --template tests/e2e/templates/many-addons.yaml.tmpl --output /tmp/kops-template422928388/manifest.yaml --values /tmp/kops-template422928388/values.yaml --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io I0321 11:21:59.332461 6418 featureflag.go:167] Unknown FeatureFlag "SpecOverrideFlag" I0321 11:21:59.446684 6386 create.go:33] /tmp/kops.x4Or16jWl create --filename /tmp/kops-template422928388/manifest.yaml --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io I0321 11:21:59.446787 6386 local.go:42] ⚙️ /tmp/kops.x4Or16jWl create --filename /tmp/kops-template422928388/manifest.yaml --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io ... skipping 62 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:22:40.789720 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:22:50.835839 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:23:00.884123 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:23:10.916706 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:23:20.948242 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:23:31.005182 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:23:41.038154 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:23:51.069162 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:24:01.102667 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:24:11.138113 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:24:21.199173 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:24:31.245445 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:24:41.280685 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:24:51.338162 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:25:01.370333 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:25:11.420045 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:25:21.465357 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:25:31.497059 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:25:41.529356 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:25:51.576159 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:26:01.628591 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:26:11.663157 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:26:21.695086 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:26:31.730372 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:26:41.780858 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:26:51.824801 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:27:01.883562 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:27:11.928294 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:27:21.958964 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:27:31.991842 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:27:42.022293 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:27:52.087119 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:28:02.122139 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:28:12.171134 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:28:22.216110 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:28:32.276320 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:28:42.309967 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0321 11:28:52.344225 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 38 lines ... Pod kube-system/ebs-csi-node-t9hg6 system-node-critical pod "ebs-csi-node-t9hg6" is pending Pod kube-system/metrics-server-db468d46c-5zz5j system-cluster-critical pod "metrics-server-db468d46c-5zz5j" is pending Pod kube-system/metrics-server-db468d46c-jzqjq system-cluster-critical pod "metrics-server-db468d46c-jzqjq" is pending Pod kube-system/pod-identity-webhook-6cd9bcbf76-5j6jc system-cluster-critical pod "pod-identity-webhook-6cd9bcbf76-5j6jc" is pending Pod kube-system/pod-identity-webhook-6cd9bcbf76-6dspk system-cluster-critical pod "pod-identity-webhook-6cd9bcbf76-6dspk" is pending Validation Failed W0321 11:29:09.824083 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 34 lines ... Pod kube-system/ebs-csi-node-gl66x system-node-critical pod "ebs-csi-node-gl66x" is pending Pod kube-system/metrics-server-db468d46c-5zz5j system-cluster-critical pod "metrics-server-db468d46c-5zz5j" is pending Pod kube-system/metrics-server-db468d46c-jzqjq system-cluster-critical pod "metrics-server-db468d46c-jzqjq" is pending Pod kube-system/pod-identity-webhook-6cd9bcbf76-5j6jc system-cluster-critical pod "pod-identity-webhook-6cd9bcbf76-5j6jc" is pending Pod kube-system/pod-identity-webhook-6cd9bcbf76-6dspk system-cluster-critical pod "pod-identity-webhook-6cd9bcbf76-6dspk" is pending Validation Failed W0321 11:29:25.827591 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 32 lines ... Pod kube-system/coredns-autoscaler-7cb5c5b969-gz88v system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-gz88v" is pending Pod kube-system/metrics-server-db468d46c-5zz5j system-cluster-critical pod "metrics-server-db468d46c-5zz5j" is pending Pod kube-system/metrics-server-db468d46c-jzqjq system-cluster-critical pod "metrics-server-db468d46c-jzqjq" is pending Pod kube-system/pod-identity-webhook-6cd9bcbf76-5j6jc system-cluster-critical pod "pod-identity-webhook-6cd9bcbf76-5j6jc" is pending Pod kube-system/pod-identity-webhook-6cd9bcbf76-6dspk system-cluster-critical pod "pod-identity-webhook-6cd9bcbf76-6dspk" is pending Validation Failed W0321 11:29:41.940659 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 70 lines ... Pod kube-system/node-local-dns-skfjg system-node-critical pod "node-local-dns-skfjg" is pending Pod kube-system/node-local-dns-wpg7h system-node-critical pod "node-local-dns-wpg7h" is pending Pod kube-system/node-local-dns-xrlrq system-node-critical pod "node-local-dns-xrlrq" is pending Pod kube-system/pod-identity-webhook-6cd9bcbf76-5j6jc system-cluster-critical pod "pod-identity-webhook-6cd9bcbf76-5j6jc" is pending Pod kube-system/pod-identity-webhook-6cd9bcbf76-6dspk system-cluster-critical pod "pod-identity-webhook-6cd9bcbf76-6dspk" is pending Validation Failed W0321 11:29:58.227984 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 59 lines ... Pod kube-system/node-local-dns-cdfld system-node-critical pod "node-local-dns-cdfld" is pending Pod kube-system/node-local-dns-nmmmc system-node-critical pod "node-local-dns-nmmmc" is pending Pod kube-system/node-local-dns-xrlrq system-node-critical pod "node-local-dns-xrlrq" is pending Pod kube-system/pod-identity-webhook-6cd9bcbf76-5j6jc system-cluster-critical pod "pod-identity-webhook-6cd9bcbf76-5j6jc" is pending Pod kube-system/pod-identity-webhook-6cd9bcbf76-6dspk system-cluster-critical pod "pod-identity-webhook-6cd9bcbf76-6dspk" is pending Validation Failed W0321 11:30:14.634190 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 39 lines ... Pod kube-system/ebs-csi-node-xxmzj system-node-critical pod "ebs-csi-node-xxmzj" is pending Pod kube-system/metrics-server-db468d46c-5zz5j system-cluster-critical pod "metrics-server-db468d46c-5zz5j" is not ready (metrics-server) Pod kube-system/metrics-server-db468d46c-jzqjq system-cluster-critical pod "metrics-server-db468d46c-jzqjq" is pending Pod kube-system/pod-identity-webhook-6cd9bcbf76-5j6jc system-cluster-critical pod "pod-identity-webhook-6cd9bcbf76-5j6jc" is pending Pod kube-system/pod-identity-webhook-6cd9bcbf76-6dspk system-cluster-critical pod "pod-identity-webhook-6cd9bcbf76-6dspk" is not ready (pod-identity-webhook) Validation Failed W0321 11:30:31.030921 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 27 lines ... Node i-0770ad67d878cfc9e control-plane node "i-0770ad67d878cfc9e" is missing kube-controller-manager pod Node i-0770ad67d878cfc9e control-plane node "i-0770ad67d878cfc9e" is missing kube-scheduler pod Pod kube-system/ebs-csi-node-pndnt system-node-critical pod "ebs-csi-node-pndnt" is pending Pod kube-system/kube-scheduler-i-0770ad67d878cfc9e system-cluster-critical pod "kube-scheduler-i-0770ad67d878cfc9e" is pending Pod kube-system/metrics-server-db468d46c-jzqjq system-cluster-critical pod "metrics-server-db468d46c-jzqjq" is not ready (metrics-server) Validation Failed W0321 11:30:47.480167 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 23 lines ... KIND NAME MESSAGE Node i-055f37bb76b47a213 control-plane node "i-055f37bb76b47a213" is missing kube-controller-manager pod Node i-055f37bb76b47a213 control-plane node "i-055f37bb76b47a213" is missing kube-scheduler pod Node i-05f05560a48588b63 control-plane node "i-05f05560a48588b63" is missing kube-scheduler pod Node i-0770ad67d878cfc9e control-plane node "i-0770ad67d878cfc9e" is missing kube-controller-manager pod Validation Failed W0321 11:31:03.895608 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 27 lines ... Node i-0770ad67d878cfc9e control-plane node "i-0770ad67d878cfc9e" is missing kube-controller-manager pod Pod kube-system/etcd-manager-events-i-055f37bb76b47a213 system-cluster-critical pod "etcd-manager-events-i-055f37bb76b47a213" is pending Pod kube-system/etcd-manager-main-i-055f37bb76b47a213 system-cluster-critical pod "etcd-manager-main-i-055f37bb76b47a213" is pending Pod kube-system/kube-proxy-i-0665f6bc169aba68f system-node-critical pod "kube-proxy-i-0665f6bc169aba68f" is pending Pod kube-system/kube-proxy-i-0fd79329e68bcb353 system-node-critical pod "kube-proxy-i-0fd79329e68bcb353" is pending Validation Failed W0321 11:31:20.908242 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 28 lines ... Pod kube-system/etcd-manager-main-i-0770ad67d878cfc9e system-cluster-critical pod "etcd-manager-main-i-0770ad67d878cfc9e" is pending Pod kube-system/kube-controller-manager-i-055f37bb76b47a213 system-cluster-critical pod "kube-controller-manager-i-055f37bb76b47a213" is pending Pod kube-system/kube-proxy-i-05f05560a48588b63 system-node-critical pod "kube-proxy-i-05f05560a48588b63" is pending Pod kube-system/kube-scheduler-i-055f37bb76b47a213 system-cluster-critical pod "kube-scheduler-i-055f37bb76b47a213" is pending Pod kube-system/kube-scheduler-i-05f05560a48588b63 system-cluster-critical pod "kube-scheduler-i-05f05560a48588b63" is pending Validation Failed W0321 11:31:39.402200 6458 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-ap-southeast-2a ControlPlane c5.large 1 1 ap-southeast-2a master-ap-southeast-2b ControlPlane c5.large 1 1 ap-southeast-2b master-ap-southeast-2c ControlPlane c5.large 1 1 ap-southeast-2c ... skipping 1236 lines ... I0321 12:06:26.026892 6571 instancegroups.go:560] Cluster did not pass validation, will retry in "30s": machine "i-0b93dc83f939c17df" has not yet joined cluster. I0321 12:07:02.702459 6571 instancegroups.go:560] Cluster did not pass validation, will retry in "30s": machine "i-0b93dc83f939c17df" has not yet joined cluster. I0321 12:07:38.938944 6571 instancegroups.go:560] Cluster did not pass validation, will retry in "30s": machine "i-0b93dc83f939c17df" has not yet joined cluster. I0321 12:08:14.824706 6571 instancegroups.go:560] Cluster did not pass validation, will retry in "30s": machine "i-0b93dc83f939c17df" has not yet joined cluster. I0321 12:08:50.767174 6571 instancegroups.go:557] Cluster did not pass validation within deadline: machine "i-0b93dc83f939c17df" has not yet joined cluster. E0321 12:08:50.767236 6571 instancegroups.go:506] Cluster did not validate within 30m0s Error: control-plane node not healthy after update, stopping rolling-update: "error validating cluster after terminating instance: cluster did not validate within a duration of \"30m0s\"" + kops-finish + kubetest2 kops -v=2 --cloud-provider=aws --cluster-name=e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --kops-root=/home/prow/go/src/k8s.io/kops --admin-access= --env=KOPS_FEATURE_FLAGS=SpecOverrideFlag --kops-binary-path=/tmp/kops.0I1XZpc5W --down I0321 12:08:50.797778 6590 featureflag.go:167] Unknown FeatureFlag "SpecOverrideFlag" I0321 12:08:50.798768 6590 app.go:61] The files in RunDir shall not be part of Artifacts I0321 12:08:50.798793 6590 app.go:62] pass rundir-in-artifacts flag True for RunDir to be part of Artifacts I0321 12:08:50.798818 6590 app.go:64] RunDir for this run: "/home/prow/go/src/k8s.io/kops/_rundir/521105f4-c7da-11ed-9e5f-baf334638f51" ... skipping 651 lines ...