e2e-kops-gce-leader-migration #1621415252220448768
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 1 failed / 1 succeeded |
| Started | |
| Elapsed | 20m12s |
| Revision | master |
exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest2 Down
... skipping 156 lines ... .............................Updated [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-12]. ..done. WARNING: No host aliases were added to your SSH configs because you do not have any running instances. Try running this command again after running some instances. I0203 07:50:26.499141 5759 up.go:44] Cleaning up any leaked resources from previous cluster I0203 07:50:26.499267 5759 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/1db491a2-a397-11ed-987c-c2f4825a9650/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh886940660/key --ssh-user prow I0203 07:50:26.499283 5759 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/1db491a2-a397-11ed-987c-c2f4825a9650/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh886940660/key --ssh-user prow Error: Cluster.kops.k8s.io "ha-migration.k8s.local" not found W0203 07:50:26.745015 5759 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0203 07:50:26.745098 5759 down.go:48] /home/prow/go/src/k8s.io/kops/_rundir/1db491a2-a397-11ed-987c-c2f4825a9650/kops delete cluster --name ha-migration.k8s.local --yes I0203 07:50:26.745122 5759 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/1db491a2-a397-11ed-987c-c2f4825a9650/kops delete cluster --name ha-migration.k8s.local --yes Error: error reading cluster configuration: Cluster.kops.k8s.io "ha-migration.k8s.local" not found I0203 07:50:26.879579 5759 gcs.go:51] gsutil ls -b -p k8s-boskos-gce-project-12 gs://k8s-boskos-gce-project-12-state-1d I0203 07:50:26.879650 5759 local.go:42] ⚙️ gsutil ls -b -p k8s-boskos-gce-project-12 gs://k8s-boskos-gce-project-12-state-1d I0203 07:50:28.922562 5759 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2023/02/03 07:50:28 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I0203 07:50:28.937965 5759 http.go:37] curl https://ip.jsb.workers.dev I0203 07:50:29.024186 5759 up.go:167] /home/prow/go/src/k8s.io/kops/_rundir/1db491a2-a397-11ed-987c-c2f4825a9650/kops create cluster --name ha-migration.k8s.local --cloud gce --kubernetes-version v1.23.16 --ssh-public-key /tmp/kops-ssh886940660/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --node-count=1 --master-count=3 --zones=us-central1-a,us-central1-b,us-central1-c --master-zones=us-central1-a,us-central1-b,us-central1-c --gce-service-account=default --admin-access 35.222.21.177/32 --master-volume-size 48 --node-volume-size 48 --master-size e2-standard-2 --project k8s-boskos-gce-project-12 I0203 07:50:29.024412 5759 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/1db491a2-a397-11ed-987c-c2f4825a9650/kops create cluster --name ha-migration.k8s.local --cloud gce --kubernetes-version v1.23.16 --ssh-public-key /tmp/kops-ssh886940660/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --node-count=1 --master-count=3 --zones=us-central1-a,us-central1-b,us-central1-c --master-zones=us-central1-a,us-central1-b,us-central1-c --gce-service-account=default --admin-access 35.222.21.177/32 --master-volume-size 48 --node-volume-size 48 --master-size e2-standard-2 --project k8s-boskos-gce-project-12 I0203 07:50:29.074675 5945 create_cluster.go:882] Using SSH public key: /tmp/kops-ssh886940660/key.pub I0203 07:50:29.201466 5945 new_cluster.go:573] VMs will be configured to use specified Service Account: default I0203 07:50:29.201587 5945 new_cluster.go:1339] Cloud Provider ID: "gce" ... skipping 527 lines ... I0203 07:50:36.540889 5966 keypair.go:226] Issuing new certificate: "etcd-peers-ca-main" W0203 07:50:36.542412 5966 vfs_keystorereader.go:143] CA private key was not found I0203 07:50:36.567385 5966 keypair.go:226] Issuing new certificate: "kubernetes-ca" I0203 07:50:36.588597 5966 keypair.go:226] Issuing new certificate: "service-account" I0203 07:50:36.610197 5966 address.go:139] GCE creating address: "api-ha-migration-k8s-local" I0203 07:50:49.276650 5966 executor.go:111] Tasks: 50 done / 91 total; 23 can run W0203 07:51:01.928250 5966 executor.go:139] error running task "ForwardingRule/api-ha-migration-k8s-local" (9m47s remaining to succeed): error creating ForwardingRule "api-ha-migration-k8s-local": googleapi: Error 400: The resource 'projects/k8s-boskos-gce-project-12/regions/us-central1/targetPools/api-ha-migration-k8s-local' is not ready, resourceNotReady I0203 07:51:01.928327 5966 executor.go:111] Tasks: 72 done / 91 total; 13 can run I0203 07:51:08.771714 5966 executor.go:111] Tasks: 85 done / 91 total; 6 can run I0203 07:51:22.696733 5966 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0203 07:51:22.841243 5966 update_cluster.go:323] Exporting kubeconfig for cluster kOps has set your kubectl context to ha-migration.k8s.local ... skipping 7 lines ... * read about installing addons at: https://kops.sigs.k8s.io/addons. I0203 07:51:33.440997 5759 up.go:251] /home/prow/go/src/k8s.io/kops/_rundir/1db491a2-a397-11ed-987c-c2f4825a9650/kops validate cluster --name ha-migration.k8s.local --count 10 --wait 15m0s I0203 07:51:33.441064 5759 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/1db491a2-a397-11ed-987c-c2f4825a9650/kops validate cluster --name ha-migration.k8s.local --count 10 --wait 15m0s Validating cluster ha-migration.k8s.local W0203 07:52:03.989675 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": dial tcp 35.184.182.59:443: i/o timeout W0203 07:52:43.990969 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": dial tcp 35.184.182.59:443: i/o timeout W0203 07:52:53.994325 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": dial tcp 35.184.182.59:443: connect: connection refused W0203 07:53:03.997729 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": dial tcp 35.184.182.59:443: connect: connection refused W0203 07:53:14.001267 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": dial tcp 35.184.182.59:443: connect: connection refused W0203 07:53:24.004178 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": dial tcp 35.184.182.59:443: connect: connection refused W0203 07:53:34.009668 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": dial tcp 35.184.182.59:443: connect: connection refused W0203 07:53:44.012997 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": dial tcp 35.184.182.59:443: connect: connection refused W0203 07:54:04.019002 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": net/http: TLS handshake timeout W0203 07:54:28.888024 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": dial tcp 35.184.182.59:443: connect: connection refused - error from a previous attempt: read tcp 10.60.124.141:43320->35.184.182.59:443: read: connection reset by peer W0203 07:54:50.900803 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": dial tcp 35.184.182.59:443: connect: connection refused - error from a previous attempt: read tcp 10.60.124.141:50430->35.184.182.59:443: read: connection reset by peer W0203 07:55:00.906037 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": dial tcp 35.184.182.59:443: connect: connection refused W0203 07:55:13.736765 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": dial tcp 35.184.182.59:443: connect: connection refused - error from a previous attempt: read tcp 10.60.124.141:51828->35.184.182.59:443: read: connection reset by peer I0203 07:55:16.402642 5759 boskos.go:86] Sending heartbeat to Boskos W0203 07:55:35.708938 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": dial tcp 35.184.182.59:443: connect: connection refused - error from a previous attempt: read tcp 10.60.124.141:35958->35.184.182.59:443: read: connection reset by peer W0203 07:55:45.714298 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": dial tcp 35.184.182.59:443: connect: connection refused W0203 07:55:55.718727 5985 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.184.182.59/api/v1/nodes": dial tcp 35.184.182.59:443: connect: connection refused I0203 07:56:14.552316 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-c ControlPlane e2-standard-2 1 1 us-central1 ... skipping 9 lines ... KIND NAME MESSAGE Machine https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-12/zones/us-central1-a/instances/nodes-us-central1-a-wpc6 machine "https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-12/zones/us-central1-a/instances/nodes-us-central1-a-wpc6" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-12/zones/us-central1-b/instances/control-plane-us-central1-b-72sx machine "https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-12/zones/us-central1-b/instances/control-plane-us-central1-b-72sx" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-12/zones/us-central1-c/instances/control-plane-us-central1-c-qkb6 machine "https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-12/zones/us-central1-c/instances/control-plane-us-central1-c-qkb6" has not yet joined cluster Node control-plane-us-central1-a-g04r node "control-plane-us-central1-a-g04r" of role "control-plane" is not ready Validation Failed W0203 07:56:15.440334 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:56:25.772359 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 11 lines ... KIND NAME MESSAGE Machine https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-12/zones/us-central1-a/instances/nodes-us-central1-a-wpc6 machine "https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-12/zones/us-central1-a/instances/nodes-us-central1-a-wpc6" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-12/zones/us-central1-b/instances/control-plane-us-central1-b-72sx machine "https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-12/zones/us-central1-b/instances/control-plane-us-central1-b-72sx" has not yet joined cluster Node control-plane-us-central1-a-g04r node "control-plane-us-central1-a-g04r" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Validation Failed W0203 07:56:26.715398 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:56:37.064223 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 20 lines ... Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-qkb6 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-qkb6" is not ready (kube-controller-manager) Pod kube-system/kube-scheduler-control-plane-us-central1-c-qkb6 system-cluster-critical pod "kube-scheduler-control-plane-us-central1-c-qkb6" is pending Validation Failed W0203 07:56:37.873347 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:56:48.261840 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 24 lines ... Pod kube-system/etcd-manager-main-control-plane-us-central1-a-g04r system-cluster-critical pod "etcd-manager-main-control-plane-us-central1-a-g04r" is pending Pod kube-system/etcd-manager-main-control-plane-us-central1-c-qkb6 system-cluster-critical pod "etcd-manager-main-control-plane-us-central1-c-qkb6" is pending Pod kube-system/kops-controller-gr46c system-cluster-critical pod "kops-controller-gr46c" is pending Pod kube-system/kops-controller-shvdg system-cluster-critical pod "kops-controller-shvdg" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-qkb6 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-qkb6" is not ready (kube-controller-manager) Validation Failed W0203 07:56:49.130968 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:56:59.488182 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 24 lines ... Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Pod kube-system/kops-controller-gr46c system-cluster-critical pod "kops-controller-gr46c" is pending Pod kube-system/kops-controller-jc2l2 system-cluster-critical pod "kops-controller-jc2l2" is pending Pod kube-system/kops-controller-shvdg system-cluster-critical pod "kops-controller-shvdg" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-qkb6 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-qkb6" is not ready (kube-controller-manager) Validation Failed W0203 07:57:00.454051 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:57:10.780980 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 23 lines ... Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Pod kube-system/kops-controller-jc2l2 system-cluster-critical pod "kops-controller-jc2l2" is pending Pod kube-system/kube-apiserver-control-plane-us-central1-c-qkb6 system-cluster-critical pod "kube-apiserver-control-plane-us-central1-c-qkb6" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-qkb6 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-qkb6" is not ready (kube-controller-manager) Validation Failed W0203 07:57:11.659550 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:57:22.100890 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 20 lines ... Pod kube-system/cilium-operator-b4b4c5b48-78cqc system-cluster-critical pod "cilium-operator-b4b4c5b48-78cqc" is pending Pod kube-system/cilium-r5pql system-node-critical pod "cilium-r5pql" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 07:57:23.043709 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:57:33.412302 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 19 lines ... Pod kube-system/cilium-operator-b4b4c5b48-78cqc system-cluster-critical pod "cilium-operator-b4b4c5b48-78cqc" is pending Pod kube-system/cilium-r5pql system-node-critical pod "cilium-r5pql" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 07:57:34.299094 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:57:44.666481 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/cilium-g5l89 system-node-critical pod "cilium-g5l89" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 07:57:45.638759 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:57:55.968136 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 07:57:56.875675 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:58:07.231469 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Pod kube-system/kube-scheduler-control-plane-us-central1-b-72sx system-cluster-critical pod "kube-scheduler-control-plane-us-central1-b-72sx" is pending Validation Failed W0203 07:58:08.100893 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:58:18.415689 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 17 lines ... Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Pod kube-system/etcd-manager-events-control-plane-us-central1-b-72sx system-cluster-critical pod "etcd-manager-events-control-plane-us-central1-b-72sx" is pending Pod kube-system/kube-apiserver-control-plane-us-central1-b-72sx system-cluster-critical pod "kube-apiserver-control-plane-us-central1-b-72sx" is pending Validation Failed W0203 07:58:19.348086 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:58:29.702061 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 07:58:30.563242 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:58:40.884401 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 07:58:41.710330 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:58:52.076996 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 07:58:52.828800 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:59:03.165381 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 07:59:04.018942 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:59:14.392475 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 07:59:15.246700 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:59:25.580084 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 07:59:26.504308 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:59:36.926974 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 07:59:37.790789 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:59:48.117753 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 07:59:48.987560 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 07:59:59.426715 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:00:00.406652 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:00:10.712786 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:00:11.574958 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:00:16.414248 5759 boskos.go:86] Sending heartbeat to Boskos I0203 08:00:21.909912 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:00:22.715365 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:00:33.003241 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:00:33.815173 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:00:44.147807 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:00:45.094259 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:00:55.534228 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:00:56.408645 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:01:06.821528 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:01:07.674288 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:01:18.028863 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:01:18.903611 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:01:29.258199 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:01:30.080954 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:01:40.527193 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:01:41.394840 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:01:51.765754 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:01:52.599412 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:02:02.896490 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:02:03.761293 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:02:14.191743 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:02:15.146799 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:02:25.769480 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:02:26.685497 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:02:37.009764 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:02:37.854728 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:02:48.213229 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:02:49.126603 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:02:59.530096 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:03:00.390198 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:03:10.672458 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:03:11.505634 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:03:21.926364 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:03:22.767028 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:03:33.151342 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:03:33.999774 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:03:44.359696 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:03:45.266146 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:03:55.603886 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:03:56.416104 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:04:06.745556 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:04:07.574163 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:04:17.958451 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:04:18.765297 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:04:29.126971 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:04:29.978888 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:04:40.310420 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:04:41.158473 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:04:51.534982 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:04:52.352756 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:05:02.706400 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:05:03.548380 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:05:13.987352 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:05:14.810961 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:05:16.426318 5759 boskos.go:86] Sending heartbeat to Boskos I0203 08:05:25.189115 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:05:26.017409 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:05:36.459325 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:05:37.321107 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:05:47.620176 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:05:48.446971 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:05:58.917452 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:05:59.718593 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:06:10.037544 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:06:10.898928 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:06:21.303910 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:06:22.084858 5985 validate_cluster.go:232] (will retry): cluster not yet healthy I0203 08:06:32.479297 5985 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-72sx node "control-plane-us-central1-b-72sx" of role "control-plane" is not ready Node control-plane-us-central1-c-qkb6 node "control-plane-us-central1-c-qkb6" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-w2vm8 system-cluster-critical pod "coredns-5d479c4f64-w2vm8" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-jmrnk system-cluster-critical pod "coredns-autoscaler-557ccb4c66-jmrnk" is pending Pod kube-system/dns-controller-6f69f47785-k4mvc system-cluster-critical pod "dns-controller-6f69f47785-k4mvc" is pending Validation Failed W0203 08:06:33.312114 5985 validate_cluster.go:232] (will retry): cluster not yet healthy Error: validation failed: wait time exceeded during validation I0203 08:06:43.321277 5759 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/1db491a2-a397-11ed-987c-c2f4825a9650/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh886940660/key --ssh-user prow I0203 08:06:43.321342 5759 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/1db491a2-a397-11ed-987c-c2f4825a9650/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh886940660/key --ssh-user prow I0203 08:06:43.714161 5996 gce.go:99] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] 2023/02/03 08:06:48 Dumping node control-plane-us-central1-a-g04r 2023/02/03 08:06:51 error dumping node control-plane-us-central1-a-g04r: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/02/03 08:06:51 error dumping node control-plane-us-central1-a-g04r: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/02/03 08:06:51 Dumping node control-plane-us-central1-b-72sx 2023/02/03 08:06:55 error dumping node control-plane-us-central1-b-72sx: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/02/03 08:06:55 error dumping node control-plane-us-central1-b-72sx: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/02/03 08:06:55 Dumping node control-plane-us-central1-c-qkb6 2023/02/03 08:07:00 error dumping node control-plane-us-central1-c-qkb6: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/02/03 08:07:00 error dumping node control-plane-us-central1-c-qkb6: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/02/03 08:07:00 dumping node not registered in kubernetes: 104.198.52.60 2023/02/03 08:07:00 Dumping node 104.198.52.60 I0203 08:07:02.113989 5759 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/1db491a2-a397-11ed-987c-c2f4825a9650/kops get cluster --name ha-migration.k8s.local -o yaml I0203 08:07:02.114024 5759 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/1db491a2-a397-11ed-987c-c2f4825a9650/kops get cluster --name ha-migration.k8s.local -o yaml I0203 08:07:02.264471 5759 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/1db491a2-a397-11ed-987c-c2f4825a9650/kops get instancegroups --name ha-migration.k8s.local -o yaml I0203 08:07:02.264511 5759 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/1db491a2-a397-11ed-987c-c2f4825a9650/kops get instancegroups --name ha-migration.k8s.local -o yaml ... skipping 166 lines ... Deleted cluster: "ha-migration.k8s.local" I0203 08:09:17.146916 5759 gcs.go:88] gsutil -u k8s-boskos-gce-project-12 rm -r gs://k8s-boskos-gce-project-12-state-1d I0203 08:09:17.146968 5759 local.go:42] ⚙️ gsutil -u k8s-boskos-gce-project-12 rm -r gs://k8s-boskos-gce-project-12-state-1d Removing gs://k8s-boskos-gce-project-12-state-1d/... I0203 08:09:20.335937 5759 down.go:62] releasing boskos project I0203 08:09:20.345504 5759 boskos.go:83] Boskos heartbeat func received signal to close Error: exit status 1 + EXIT_VALUE=1 + set +o xtrace