e2e-kops-aws-upgrade-kstable-kolatest-to-kci-kolatest-many-addons #1636326631150718976
Recent runs || View in Spyglass
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 0 failed / 1 succeeded |
| Started | |
| Elapsed | 53m0s |
| Revision | master |
kubetest2 Down
... skipping 205 lines ... I0316 11:22:40.045146 6318 app.go:64] RunDir for this run: "/home/prow/go/src/k8s.io/kops/_rundir/98193b54-c3ec-11ed-93d6-92b4ce3fddda" I0316 11:22:40.047954 6318 app.go:128] ID for this run: "98193b54-c3ec-11ed-93d6-92b4ce3fddda" I0316 11:22:40.048125 6318 local.go:42] ⚙️ ssh-keygen -t ed25519 -N -q -f /tmp/kops/e2e-9b8b75964a-36844.test-cncf-aws.k8s.io/id_ed25519 I0316 11:22:40.056041 6318 dumplogs.go:45] /tmp/kops.7FVQqk5c5 toolbox dump --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /tmp/kops/e2e-9b8b75964a-36844.test-cncf-aws.k8s.io/id_ed25519 --ssh-user ubuntu I0316 11:22:40.056081 6318 local.go:42] ⚙️ /tmp/kops.7FVQqk5c5 toolbox dump --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /tmp/kops/e2e-9b8b75964a-36844.test-cncf-aws.k8s.io/id_ed25519 --ssh-user ubuntu I0316 11:22:40.087296 6328 featureflag.go:167] Unknown FeatureFlag "SpecOverrideFlag" Error: Cluster.kops.k8s.io "e2e-9b8b75964a-36844.test-cncf-aws.k8s.io" not found W0316 11:22:40.536401 6318 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0316 11:22:40.536455 6318 down.go:48] /tmp/kops.7FVQqk5c5 delete cluster --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --yes I0316 11:22:40.536479 6318 local.go:42] ⚙️ /tmp/kops.7FVQqk5c5 delete cluster --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --yes I0316 11:22:40.566438 6337 featureflag.go:167] Unknown FeatureFlag "SpecOverrideFlag" Error: error reading cluster configuration: Cluster.kops.k8s.io "e2e-9b8b75964a-36844.test-cncf-aws.k8s.io" not found Error: exit status 1 + echo 'kubetest2 down failed' kubetest2 down failed + [[ l == \v ]] ++ kops-base-from-marker latest ++ [[ latest =~ ^https: ]] ++ [[ latest == \l\a\t\e\s\t ]] ++ curl -fs https://storage.googleapis.com/kops-ci/bin/latest-ci-updown-green.txt + KOPS_BASE_URL=https://storage.googleapis.com/kops-ci/bin/1.27.0-alpha.2+v1.27.0-alpha.1-452-g071f9a86ad ... skipping 15 lines ... I0316 11:22:42.561939 6373 app.go:64] RunDir for this run: "/home/prow/go/src/k8s.io/kops/_rundir/98193b54-c3ec-11ed-93d6-92b4ce3fddda" I0316 11:22:42.566901 6373 app.go:128] ID for this run: "98193b54-c3ec-11ed-93d6-92b4ce3fddda" I0316 11:22:42.566950 6373 up.go:44] Cleaning up any leaked resources from previous cluster I0316 11:22:42.566978 6373 dumplogs.go:45] /tmp/kops.9RA3eUQsG toolbox dump --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /tmp/kops/e2e-9b8b75964a-36844.test-cncf-aws.k8s.io/id_ed25519 --ssh-user ubuntu I0316 11:22:42.567011 6373 local.go:42] ⚙️ /tmp/kops.9RA3eUQsG toolbox dump --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /tmp/kops/e2e-9b8b75964a-36844.test-cncf-aws.k8s.io/id_ed25519 --ssh-user ubuntu I0316 11:22:42.596995 6381 featureflag.go:167] Unknown FeatureFlag "SpecOverrideFlag" Error: Cluster.kops.k8s.io "e2e-9b8b75964a-36844.test-cncf-aws.k8s.io" not found W0316 11:22:43.072025 6373 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0316 11:22:43.072064 6373 down.go:48] /tmp/kops.9RA3eUQsG delete cluster --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --yes I0316 11:22:43.072077 6373 local.go:42] ⚙️ /tmp/kops.9RA3eUQsG delete cluster --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --yes I0316 11:22:43.100054 6393 featureflag.go:167] Unknown FeatureFlag "SpecOverrideFlag" Error: error reading cluster configuration: Cluster.kops.k8s.io "e2e-9b8b75964a-36844.test-cncf-aws.k8s.io" not found I0316 11:22:43.494842 6373 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2023/03/16 11:22:43 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I0316 11:22:43.506693 6373 http.go:37] curl https://ip.jsb.workers.dev I0316 11:22:43.611516 6373 template.go:58] /tmp/kops.9RA3eUQsG toolbox template --template tests/e2e/templates/many-addons.yaml.tmpl --output /tmp/kops-template2998104945/manifest.yaml --values /tmp/kops-template2998104945/values.yaml --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io I0316 11:22:43.611553 6373 local.go:42] ⚙️ /tmp/kops.9RA3eUQsG toolbox template --template tests/e2e/templates/many-addons.yaml.tmpl --output /tmp/kops-template2998104945/manifest.yaml --values /tmp/kops-template2998104945/values.yaml --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io I0316 11:22:43.641100 6404 featureflag.go:167] Unknown FeatureFlag "SpecOverrideFlag" I0316 11:22:43.731492 6373 create.go:33] /tmp/kops.9RA3eUQsG create --filename /tmp/kops-template2998104945/manifest.yaml --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io I0316 11:22:43.731537 6373 local.go:42] ⚙️ /tmp/kops.9RA3eUQsG create --filename /tmp/kops-template2998104945/manifest.yaml --name e2e-9b8b75964a-36844.test-cncf-aws.k8s.io ... skipping 61 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:23:21.368488 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:23:31.411377 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:23:41.443688 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:23:51.472512 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:24:01.503309 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:24:11.531173 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:24:21.566171 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:24:31.605724 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:24:41.638730 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:24:51.680769 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:25:01.712695 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:25:11.752291 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:25:21.785357 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:25:31.815797 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:25:41.847393 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:25:51.889449 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:26:01.947636 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:26:11.977517 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:26:22.009056 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:26:32.043022 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:26:42.102583 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:26:52.132631 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:27:02.174860 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:27:12.211379 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:27:22.241541 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:27:32.271648 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:27:42.327475 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:27:52.361715 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:28:02.390882 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:28:12.450184 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:28:22.484806 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:28:32.514953 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:28:42.557450 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:28:52.615686 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:29:02.659296 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:29:12.714187 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:29:22.753431 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:29:32.782291 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:29:42.810614 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 3 lines ... NODE STATUS NAME ROLE READY VALIDATION ERRORS KIND NAME MESSAGE dns apiserver Validation Failed The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address. The API DNS IP address is the placeholder address that kops creates: 203.0.113.123. Please wait about 5-10 minutes for a control plane node to start, dns-controller to launch, and DNS to propagate. The protokube container and dns-controller deployment logs may contain more diagnostic information. Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start. Validation Failed W0316 11:29:52.864769 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 43 lines ... Pod kube-system/metrics-server-db468d46c-mg8kc system-cluster-critical pod "metrics-server-db468d46c-mg8kc" is pending Pod kube-system/metrics-server-db468d46c-wpwbm system-cluster-critical pod "metrics-server-db468d46c-wpwbm" is pending Pod kube-system/node-local-dns-29x58 system-node-critical pod "node-local-dns-29x58" is pending Pod kube-system/pod-identity-webhook-676bbc5847-n84bp system-cluster-critical pod "pod-identity-webhook-676bbc5847-n84bp" is pending Pod kube-system/pod-identity-webhook-676bbc5847-qvspj system-cluster-critical pod "pod-identity-webhook-676bbc5847-qvspj" is pending Validation Failed W0316 11:30:09.288387 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 37 lines ... Pod kube-system/metrics-server-db468d46c-mg8kc system-cluster-critical pod "metrics-server-db468d46c-mg8kc" is pending Pod kube-system/metrics-server-db468d46c-wpwbm system-cluster-critical pod "metrics-server-db468d46c-wpwbm" is pending Pod kube-system/node-local-dns-29x58 system-node-critical pod "node-local-dns-29x58" is pending Pod kube-system/pod-identity-webhook-676bbc5847-n84bp system-cluster-critical pod "pod-identity-webhook-676bbc5847-n84bp" is pending Pod kube-system/pod-identity-webhook-676bbc5847-qvspj system-cluster-critical pod "pod-identity-webhook-676bbc5847-qvspj" is pending Validation Failed W0316 11:30:24.515551 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 70 lines ... Pod kube-system/node-local-dns-k7zw9 system-node-critical pod "node-local-dns-k7zw9" is pending Pod kube-system/node-local-dns-khzwf system-node-critical pod "node-local-dns-khzwf" is pending Pod kube-system/node-local-dns-xhtdt system-node-critical pod "node-local-dns-xhtdt" is pending Pod kube-system/pod-identity-webhook-676bbc5847-n84bp system-cluster-critical pod "pod-identity-webhook-676bbc5847-n84bp" is pending Pod kube-system/pod-identity-webhook-676bbc5847-qvspj system-cluster-critical pod "pod-identity-webhook-676bbc5847-qvspj" is pending Validation Failed W0316 11:30:39.674227 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 64 lines ... Pod kube-system/node-local-dns-4vzq8 system-node-critical pod "node-local-dns-4vzq8" is pending Pod kube-system/node-local-dns-dclwt system-node-critical pod "node-local-dns-dclwt" is pending Pod kube-system/node-local-dns-t8hl6 system-node-critical pod "node-local-dns-t8hl6" is pending Pod kube-system/pod-identity-webhook-676bbc5847-n84bp system-cluster-critical pod "pod-identity-webhook-676bbc5847-n84bp" is pending Pod kube-system/pod-identity-webhook-676bbc5847-qvspj system-cluster-critical pod "pod-identity-webhook-676bbc5847-qvspj" is pending Validation Failed W0316 11:30:54.379580 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 46 lines ... Pod kube-system/ebs-csi-node-wljkb system-node-critical pod "ebs-csi-node-wljkb" is pending Pod kube-system/metrics-server-db468d46c-mg8kc system-cluster-critical pod "metrics-server-db468d46c-mg8kc" is pending Pod kube-system/metrics-server-db468d46c-wpwbm system-cluster-critical pod "metrics-server-db468d46c-wpwbm" is not ready (metrics-server) Pod kube-system/pod-identity-webhook-676bbc5847-n84bp system-cluster-critical pod "pod-identity-webhook-676bbc5847-n84bp" is pending Pod kube-system/pod-identity-webhook-676bbc5847-qvspj system-cluster-critical pod "pod-identity-webhook-676bbc5847-qvspj" is pending Validation Failed W0316 11:31:09.116543 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 34 lines ... Pod kube-system/ebs-csi-node-6gt72 system-node-critical pod "ebs-csi-node-6gt72" is pending Pod kube-system/kube-apiserver-i-060a9ba1038130825 system-cluster-critical pod "kube-apiserver-i-060a9ba1038130825" is pending Pod kube-system/metrics-server-db468d46c-mg8kc system-cluster-critical pod "metrics-server-db468d46c-mg8kc" is not ready (metrics-server) Pod kube-system/metrics-server-db468d46c-wpwbm system-cluster-critical pod "metrics-server-db468d46c-wpwbm" is not ready (metrics-server) Pod kube-system/pod-identity-webhook-676bbc5847-qvspj system-cluster-critical pod "pod-identity-webhook-676bbc5847-qvspj" is pending Validation Failed W0316 11:31:23.838948 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 29 lines ... Node i-0dec9bb648cce0e5e control-plane node "i-0dec9bb648cce0e5e" is missing kube-scheduler pod Pod kube-system/aws-load-balancer-controller-56c4d5b99c-8t5s9 system-cluster-critical pod "aws-load-balancer-controller-56c4d5b99c-8t5s9" is pending Pod kube-system/aws-load-balancer-controller-56c4d5b99c-l722r system-cluster-critical pod "aws-load-balancer-controller-56c4d5b99c-l722r" is pending Pod kube-system/etcd-manager-events-i-060a9ba1038130825 system-cluster-critical pod "etcd-manager-events-i-060a9ba1038130825" is pending Pod kube-system/pod-identity-webhook-676bbc5847-qvspj system-cluster-critical pod "pod-identity-webhook-676bbc5847-qvspj" is pending Validation Failed W0316 11:31:38.763919 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 29 lines ... Pod kube-system/aws-load-balancer-controller-56c4d5b99c-8t5s9 system-cluster-critical pod "aws-load-balancer-controller-56c4d5b99c-8t5s9" is pending Pod kube-system/aws-load-balancer-controller-56c4d5b99c-l722r system-cluster-critical pod "aws-load-balancer-controller-56c4d5b99c-l722r" is pending Pod kube-system/etcd-manager-main-i-0dec9bb648cce0e5e system-cluster-critical pod "etcd-manager-main-i-0dec9bb648cce0e5e" is pending Pod kube-system/kube-proxy-i-0dec9bb648cce0e5e system-node-critical pod "kube-proxy-i-0dec9bb648cce0e5e" is pending Pod kube-system/kube-scheduler-i-01380c267b7d16225 system-cluster-critical pod "kube-scheduler-i-01380c267b7d16225" is pending Validation Failed W0316 11:31:53.560456 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 26 lines ... Pod kube-system/kube-proxy-i-060a9ba1038130825 system-node-critical pod "kube-proxy-i-060a9ba1038130825" is pending Pod kube-system/kube-proxy-i-0aa7df499889442b8 system-node-critical pod "kube-proxy-i-0aa7df499889442b8" is pending Pod kube-system/kube-proxy-i-0b44179ac04be2d67 system-node-critical pod "kube-proxy-i-0b44179ac04be2d67" is pending Pod kube-system/kube-proxy-i-0d75d2740eba97de0 system-node-critical pod "kube-proxy-i-0d75d2740eba97de0" is pending Pod kube-system/kube-proxy-i-0d913d56f605d30ce system-node-critical pod "kube-proxy-i-0d913d56f605d30ce" is pending Validation Failed W0316 11:32:08.817237 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 23 lines ... KIND NAME MESSAGE Node i-060a9ba1038130825 control-plane node "i-060a9ba1038130825" is missing kube-scheduler pod Node i-0dec9bb648cce0e5e control-plane node "i-0dec9bb648cce0e5e" is missing kube-scheduler pod Pod kube-system/kube-scheduler-i-060a9ba1038130825 system-cluster-critical pod "kube-scheduler-i-060a9ba1038130825" is pending Pod kube-system/kube-scheduler-i-0dec9bb648cce0e5e system-cluster-critical pod "kube-scheduler-i-0dec9bb648cce0e5e" is pending Validation Failed W0316 11:32:23.665163 6442 validate_cluster.go:232] (will retry): cluster not yet healthy INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-sa-east-1a ControlPlane c5.large 1 1 sa-east-1a master-sa-east-1b ControlPlane c5.large 1 1 sa-east-1b master-sa-east-1c ControlPlane c5.large 1 1 sa-east-1c ... skipping 1217 lines ... I0316 12:06:32.215747 6556 instancegroups.go:560] Cluster did not pass validation, will retry in "30s": machine "i-0a4670c9fbbf93f00" has not yet joined cluster. I0316 12:07:07.137730 6556 instancegroups.go:560] Cluster did not pass validation, will retry in "30s": machine "i-0a4670c9fbbf93f00" has not yet joined cluster. I0316 12:07:42.346566 6556 instancegroups.go:560] Cluster did not pass validation, will retry in "30s": machine "i-0a4670c9fbbf93f00" has not yet joined cluster. I0316 12:08:17.323509 6556 instancegroups.go:560] Cluster did not pass validation, will retry in "30s": machine "i-0a4670c9fbbf93f00" has not yet joined cluster. I0316 12:08:52.550596 6556 instancegroups.go:557] Cluster did not pass validation within deadline: machine "i-0a4670c9fbbf93f00" has not yet joined cluster. E0316 12:08:52.550644 6556 instancegroups.go:506] Cluster did not validate within 30m0s Error: control-plane node not healthy after update, stopping rolling-update: "error validating cluster after terminating instance: cluster did not validate within a duration of \"30m0s\"" + kops-finish + kubetest2 kops -v=2 --cloud-provider=aws --cluster-name=e2e-9b8b75964a-36844.test-cncf-aws.k8s.io --kops-root=/home/prow/go/src/k8s.io/kops --admin-access= --env=KOPS_FEATURE_FLAGS=SpecOverrideFlag --kops-binary-path=/tmp/kops.7FVQqk5c5 --down I0316 12:08:52.578674 6574 featureflag.go:167] Unknown FeatureFlag "SpecOverrideFlag" I0316 12:08:52.579839 6574 app.go:61] The files in RunDir shall not be part of Artifacts I0316 12:08:52.579934 6574 app.go:62] pass rundir-in-artifacts flag True for RunDir to be part of Artifacts I0316 12:08:52.579993 6574 app.go:64] RunDir for this run: "/home/prow/go/src/k8s.io/kops/_rundir/98193b54-c3ec-11ed-93d6-92b4ce3fddda" ... skipping 563 lines ...