e2e-kops-gce-leader-migration #1620599979334701056
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 1 failed / 1 succeeded |
| Started | |
| Elapsed | 20m14s |
| Revision | master |
exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest2 Down
... skipping 156 lines ... ........................................Updated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2]. ..done. WARNING: No host aliases were added to your SSH configs because you do not have any running instances. Try running this command again after running some instances. I0201 01:50:34.839138 5771 up.go:44] Cleaning up any leaked resources from previous cluster I0201 01:50:34.839261 5771 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/8c760572-a1d2-11ed-9efc-5e7820dba16b/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh2453924106/key --ssh-user prow I0201 01:50:34.839276 5771 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/8c760572-a1d2-11ed-9efc-5e7820dba16b/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh2453924106/key --ssh-user prow Error: Cluster.kops.k8s.io "ha-migration.k8s.local" not found W0201 01:50:35.156825 5771 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0201 01:50:35.156872 5771 down.go:48] /home/prow/go/src/k8s.io/kops/_rundir/8c760572-a1d2-11ed-9efc-5e7820dba16b/kops delete cluster --name ha-migration.k8s.local --yes I0201 01:50:35.156886 5771 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/8c760572-a1d2-11ed-9efc-5e7820dba16b/kops delete cluster --name ha-migration.k8s.local --yes Error: error reading cluster configuration: Cluster.kops.k8s.io "ha-migration.k8s.local" not found I0201 01:50:35.381265 5771 gcs.go:51] gsutil ls -b -p k8s-jkns-e2e-gce-reboot-1-2 gs://k8s-jkns-e2e-gce-reboot-1-2-state-8c I0201 01:50:35.381300 5771 local.go:42] ⚙️ gsutil ls -b -p k8s-jkns-e2e-gce-reboot-1-2 gs://k8s-jkns-e2e-gce-reboot-1-2-state-8c I0201 01:50:37.252595 5771 gcs.go:70] gsutil mb -p k8s-jkns-e2e-gce-reboot-1-2 gs://k8s-jkns-e2e-gce-reboot-1-2-state-8c I0201 01:50:37.252626 5771 local.go:42] ⚙️ gsutil mb -p k8s-jkns-e2e-gce-reboot-1-2 gs://k8s-jkns-e2e-gce-reboot-1-2-state-8c Creating gs://k8s-jkns-e2e-gce-reboot-1-2-state-8c/... I0201 01:50:39.613268 5771 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2023/02/01 01:50:39 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I0201 01:50:39.626702 5771 http.go:37] curl https://ip.jsb.workers.dev I0201 01:50:39.733356 5771 up.go:167] /home/prow/go/src/k8s.io/kops/_rundir/8c760572-a1d2-11ed-9efc-5e7820dba16b/kops create cluster --name ha-migration.k8s.local --cloud gce --kubernetes-version v1.23.16 --ssh-public-key /tmp/kops-ssh2453924106/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --node-count=1 --master-count=3 --zones=us-central1-a,us-central1-b,us-central1-c --master-zones=us-central1-a,us-central1-b,us-central1-c --gce-service-account=default --admin-access 34.136.218.137/32 --master-volume-size 48 --node-volume-size 48 --master-size e2-standard-2 --project k8s-jkns-e2e-gce-reboot-1-2 I0201 01:50:39.733400 5771 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/8c760572-a1d2-11ed-9efc-5e7820dba16b/kops create cluster --name ha-migration.k8s.local --cloud gce --kubernetes-version v1.23.16 --ssh-public-key /tmp/kops-ssh2453924106/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --node-count=1 --master-count=3 --zones=us-central1-a,us-central1-b,us-central1-c --master-zones=us-central1-a,us-central1-b,us-central1-c --gce-service-account=default --admin-access 34.136.218.137/32 --master-volume-size 48 --node-volume-size 48 --master-size e2-standard-2 --project k8s-jkns-e2e-gce-reboot-1-2 I0201 01:50:39.780364 6091 create_cluster.go:882] Using SSH public key: /tmp/kops-ssh2453924106/key.pub I0201 01:50:40.042571 6091 new_cluster.go:573] VMs will be configured to use specified Service Account: default I0201 01:50:40.042730 6091 new_cluster.go:1339] Cloud Provider ID: "gce" ... skipping 527 lines ... I0201 01:50:52.376913 6111 keypair.go:226] Issuing new certificate: "etcd-manager-ca-main" I0201 01:50:52.386488 6111 keypair.go:226] Issuing new certificate: "etcd-peers-ca-events" W0201 01:50:52.578777 6111 vfs_keystorereader.go:143] CA private key was not found I0201 01:50:52.675244 6111 keypair.go:226] Issuing new certificate: "service-account" I0201 01:50:52.770622 6111 keypair.go:226] Issuing new certificate: "kubernetes-ca" I0201 01:51:04.865146 6111 executor.go:111] Tasks: 50 done / 91 total; 23 can run W0201 01:51:17.875413 6111 executor.go:139] error running task "ForwardingRule/api-ha-migration-k8s-local" (9m46s remaining to succeed): error creating ForwardingRule "api-ha-migration-k8s-local": googleapi: Error 400: The resource 'projects/k8s-jkns-e2e-gce-reboot-1-2/regions/us-central1/targetPools/api-ha-migration-k8s-local' is not ready, resourceNotReady I0201 01:51:17.875477 6111 executor.go:111] Tasks: 72 done / 91 total; 13 can run I0201 01:51:24.755866 6111 executor.go:111] Tasks: 85 done / 91 total; 6 can run I0201 01:51:41.802655 6111 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0201 01:51:41.872674 6111 update_cluster.go:323] Exporting kubeconfig for cluster kOps has set your kubectl context to ha-migration.k8s.local ... skipping 7 lines ... * read about installing addons at: https://kops.sigs.k8s.io/addons. I0201 01:51:52.101640 5771 up.go:251] /home/prow/go/src/k8s.io/kops/_rundir/8c760572-a1d2-11ed-9efc-5e7820dba16b/kops validate cluster --name ha-migration.k8s.local --count 10 --wait 15m0s I0201 01:51:52.101702 5771 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/8c760572-a1d2-11ed-9efc-5e7820dba16b/kops validate cluster --name ha-migration.k8s.local --count 10 --wait 15m0s Validating cluster ha-migration.k8s.local W0201 01:52:22.574949 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: i/o timeout W0201 01:53:02.576656 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: i/o timeout W0201 01:53:12.580651 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: connect: connection refused W0201 01:53:22.583847 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: connect: connection refused W0201 01:53:32.586782 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: connect: connection refused W0201 01:53:42.589323 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: connect: connection refused W0201 01:53:52.595085 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: connect: connection refused W0201 01:54:02.598275 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: connect: connection refused W0201 01:54:22.601836 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": net/http: TLS handshake timeout W0201 01:54:40.998152 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: connect: connection refused - error from a previous attempt: read tcp 10.60.181.28:60288->104.198.224.206:443: read: connection reset by peer W0201 01:55:01.010799 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": net/http: TLS handshake timeout W0201 01:55:11.013425 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: connect: connection refused W0201 01:55:21.016095 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: connect: connection refused I0201 01:55:23.359276 5771 boskos.go:86] Sending heartbeat to Boskos W0201 01:55:31.018625 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: connect: connection refused W0201 01:55:51.021744 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": net/http: TLS handshake timeout W0201 01:56:01.024820 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: connect: connection refused W0201 01:56:11.028622 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: connect: connection refused W0201 01:56:21.031348 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: connect: connection refused W0201 01:56:46.070098 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": net/http: TLS handshake timeout - error from a previous attempt: read tcp 10.60.181.28:51148->104.198.224.206:443: read: connection reset by peer W0201 01:56:56.077418 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: connect: connection refused W0201 01:57:06.081780 6130 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://104.198.224.206/api/v1/nodes": dial tcp 104.198.224.206:443: connect: connection refused I0201 01:57:16.612521 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-c ControlPlane e2-standard-2 1 1 us-central1 ... skipping 10 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-a/instances/control-plane-us-central1-a-q1bh machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-a/instances/control-plane-us-central1-a-q1bh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-a/instances/nodes-us-central1-a-p77z machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-a/instances/nodes-us-central1-a-p77z" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-c/instances/control-plane-us-central1-c-cjj4 machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-c/instances/control-plane-us-central1-c-cjj4" has not yet joined cluster Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Pod kube-system/kube-controller-manager-control-plane-us-central1-b-zggc system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-b-zggc" is not ready (kube-controller-manager) Validation Failed W0201 01:57:17.525476 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 01:57:28.003839 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 11 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-a/instances/control-plane-us-central1-a-q1bh machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-a/instances/control-plane-us-central1-a-q1bh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-a/instances/nodes-us-central1-a-p77z machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-a/instances/nodes-us-central1-a-p77z" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-c/instances/control-plane-us-central1-c-cjj4 machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-c/instances/control-plane-us-central1-c-cjj4" has not yet joined cluster Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Pod kube-system/kube-controller-manager-control-plane-us-central1-b-zggc system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-b-zggc" is not ready (kube-controller-manager) Validation Failed W0201 01:57:28.996849 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 01:57:39.441081 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 11 lines ... KIND NAME MESSAGE Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-a/instances/nodes-us-central1-a-p77z machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-a/instances/nodes-us-central1-a-p77z" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-c/instances/control-plane-us-central1-c-cjj4 machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-reboot-1-2/zones/us-central1-c/instances/control-plane-us-central1-c-cjj4" has not yet joined cluster Node control-plane-us-central1-a-q1bh node "control-plane-us-central1-a-q1bh" of role "control-plane" is not ready Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Validation Failed W0201 01:57:40.224760 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 01:57:50.691746 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 20 lines ... Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Pod kube-system/kops-controller-phx6f system-cluster-critical pod "kops-controller-phx6f" is pending Pod kube-system/kops-controller-zvjtw system-cluster-critical pod "kops-controller-zvjtw" is pending Validation Failed W0201 01:57:51.553740 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 01:58:01.967217 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 22 lines ... Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Pod kube-system/etcd-manager-events-control-plane-us-central1-b-zggc system-cluster-critical pod "etcd-manager-events-control-plane-us-central1-b-zggc" is pending Pod kube-system/kops-controller-zvjtw system-cluster-critical pod "kops-controller-zvjtw" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-cjj4 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-cjj4" is pending Validation Failed W0201 01:58:02.810822 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 01:58:13.411227 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 19 lines ... Pod kube-system/cilium-zkpdm system-node-critical pod "cilium-zkpdm" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-cjj4 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-cjj4" is not ready (kube-controller-manager) Validation Failed W0201 01:58:14.260633 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 01:58:24.684836 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 19 lines ... Pod kube-system/cilium-zkpdm system-node-critical pod "cilium-zkpdm" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-cjj4 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-cjj4" is not ready (kube-controller-manager) Validation Failed W0201 01:58:25.515482 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 01:58:35.975286 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 19 lines ... Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Pod kube-system/kube-apiserver-control-plane-us-central1-a-q1bh system-cluster-critical pod "kube-apiserver-control-plane-us-central1-a-q1bh" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-cjj4 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-cjj4" is not ready (kube-controller-manager) Validation Failed W0201 01:58:36.819179 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 01:58:48.347572 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 17 lines ... Pod kube-system/cilium-hwq4t system-node-critical pod "cilium-hwq4t" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-cjj4 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-cjj4" is not ready (kube-controller-manager) Validation Failed W0201 01:58:49.282584 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 01:58:59.699152 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 19 lines ... Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Pod kube-system/kube-apiserver-control-plane-us-central1-c-cjj4 system-cluster-critical pod "kube-apiserver-control-plane-us-central1-c-cjj4" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-cjj4 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-cjj4" is not ready (kube-controller-manager) Pod kube-system/kube-scheduler-control-plane-us-central1-c-cjj4 system-cluster-critical pod "kube-scheduler-control-plane-us-central1-c-cjj4" is pending Validation Failed W0201 01:59:00.528684 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 01:59:11.047665 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-cjj4 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-cjj4" is not ready (kube-controller-manager) Validation Failed W0201 01:59:11.947750 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 01:59:22.394089 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-cjj4 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-cjj4" is not ready (kube-controller-manager) Validation Failed W0201 01:59:23.209982 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 01:59:33.601830 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-cjj4 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-cjj4" is not ready (kube-controller-manager) Validation Failed W0201 01:59:34.481503 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 01:59:44.927698 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-cjj4 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-cjj4" is not ready (kube-controller-manager) Validation Failed W0201 01:59:45.731276 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 01:59:56.150645 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-cjj4 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-cjj4" is not ready (kube-controller-manager) Validation Failed W0201 01:59:57.008500 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:00:07.490261 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-cjj4 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-cjj4" is not ready (kube-controller-manager) Validation Failed W0201 02:00:08.262596 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:00:18.689682 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:00:19.572109 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:00:23.368749 5771 boskos.go:86] Sending heartbeat to Boskos I0201 02:00:30.069160 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:00:30.942263 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:00:41.401160 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:00:42.224447 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:00:52.633776 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:00:53.507944 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:01:03.925542 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:01:04.814839 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:01:15.326415 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:01:16.140937 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:01:26.553916 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:01:27.393077 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:01:37.852596 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:01:38.707122 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:01:49.187329 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:01:50.117771 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:02:00.580442 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:02:01.423727 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:02:11.926413 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:02:12.834281 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:02:23.227720 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:02:24.152467 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:02:34.626241 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:02:35.488076 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:02:45.956793 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:02:46.822256 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:02:57.244628 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:02:58.101160 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:03:08.526123 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:03:09.612293 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:03:20.077762 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:03:21.067847 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:03:31.490355 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:03:32.369916 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:03:42.782733 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:03:43.636834 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:03:54.076254 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:03:55.017836 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:04:05.466943 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:04:06.342914 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:04:16.788326 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:04:17.663163 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:04:28.185940 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:04:29.075814 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:04:39.514330 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:04:40.312427 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:04:50.789041 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:04:51.637902 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:05:02.143371 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:05:02.966692 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:05:13.402468 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:05:14.243862 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:05:23.382085 5771 boskos.go:86] Sending heartbeat to Boskos I0201 02:05:24.694251 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:05:25.527697 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:05:35.950345 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:05:36.842476 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:05:47.305083 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:05:48.110550 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:05:58.552559 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:05:59.444903 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:06:09.927565 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:06:10.770089 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:06:21.222250 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:06:22.014931 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:06:32.446007 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:06:33.279043 6130 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 02:06:43.741826 6130 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-zggc node "control-plane-us-central1-b-zggc" of role "control-plane" is not ready Node control-plane-us-central1-c-cjj4 node "control-plane-us-central1-c-cjj4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-5l9jz system-cluster-critical pod "coredns-5d479c4f64-5l9jz" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-ldr69 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-ldr69" is pending Pod kube-system/dns-controller-6f69f47785-krpqn system-cluster-critical pod "dns-controller-6f69f47785-krpqn" is pending Validation Failed W0201 02:06:44.628184 6130 validate_cluster.go:232] (will retry): cluster not yet healthy Error: validation failed: wait time exceeded during validation I0201 02:06:54.637741 5771 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/8c760572-a1d2-11ed-9efc-5e7820dba16b/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh2453924106/key --ssh-user prow I0201 02:06:54.637795 5771 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/8c760572-a1d2-11ed-9efc-5e7820dba16b/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh2453924106/key --ssh-user prow I0201 02:06:55.082813 6140 gce.go:99] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] 2023/02/01 02:06:59 Dumping node control-plane-us-central1-a-q1bh 2023/02/01 02:07:04 error dumping node control-plane-us-central1-a-q1bh: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/02/01 02:07:04 error dumping node control-plane-us-central1-a-q1bh: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/02/01 02:07:04 Dumping node control-plane-us-central1-b-zggc 2023/02/01 02:07:07 error dumping node control-plane-us-central1-b-zggc: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/02/01 02:07:07 error dumping node control-plane-us-central1-b-zggc: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/02/01 02:07:07 Dumping node control-plane-us-central1-c-cjj4 2023/02/01 02:07:10 error dumping node control-plane-us-central1-c-cjj4: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/02/01 02:07:10 error dumping node control-plane-us-central1-c-cjj4: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/02/01 02:07:10 dumping node not registered in kubernetes: 34.69.17.18 2023/02/01 02:07:10 Dumping node 34.69.17.18 I0201 02:07:12.218927 5771 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/8c760572-a1d2-11ed-9efc-5e7820dba16b/kops get cluster --name ha-migration.k8s.local -o yaml I0201 02:07:12.218962 5771 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/8c760572-a1d2-11ed-9efc-5e7820dba16b/kops get cluster --name ha-migration.k8s.local -o yaml I0201 02:07:12.371473 5771 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/8c760572-a1d2-11ed-9efc-5e7820dba16b/kops get instancegroups --name ha-migration.k8s.local -o yaml I0201 02:07:12.371520 5771 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/8c760572-a1d2-11ed-9efc-5e7820dba16b/kops get instancegroups --name ha-migration.k8s.local -o yaml ... skipping 166 lines ... Deleted cluster: "ha-migration.k8s.local" I0201 02:09:32.999727 5771 gcs.go:88] gsutil -u k8s-jkns-e2e-gce-reboot-1-2 rm -r gs://k8s-jkns-e2e-gce-reboot-1-2-state-8c I0201 02:09:32.999776 5771 local.go:42] ⚙️ gsutil -u k8s-jkns-e2e-gce-reboot-1-2 rm -r gs://k8s-jkns-e2e-gce-reboot-1-2-state-8c Removing gs://k8s-jkns-e2e-gce-reboot-1-2-state-8c/... I0201 02:09:35.365202 5771 down.go:62] releasing boskos project I0201 02:09:35.373638 5771 boskos.go:83] Boskos heartbeat func received signal to close Error: exit status 1 + EXIT_VALUE=1 + set +o xtrace