e2e-kops-gce-leader-migration #1621279509795835904
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 1 failed / 1 succeeded |
| Started | |
| Elapsed | 20m26s |
| Revision | master |
exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest2 Down
... skipping 156 lines ... .....................................Updated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-2]. .done. WARNING: No host aliases were added to your SSH configs because you do not have any running instances. Try running this command again after running some instances. I0202 22:51:00.882889 5790 up.go:44] Cleaning up any leaked resources from previous cluster I0202 22:51:00.883037 5790 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/c38c579c-a34b-11ed-8c0a-1250e2f4dc61/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh4289614941/key --ssh-user prow I0202 22:51:00.883053 5790 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/c38c579c-a34b-11ed-8c0a-1250e2f4dc61/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh4289614941/key --ssh-user prow Error: Cluster.kops.k8s.io "ha-migration.k8s.local" not found W0202 22:51:01.098408 5790 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0202 22:51:01.098451 5790 down.go:48] /home/prow/go/src/k8s.io/kops/_rundir/c38c579c-a34b-11ed-8c0a-1250e2f4dc61/kops delete cluster --name ha-migration.k8s.local --yes I0202 22:51:01.098462 5790 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/c38c579c-a34b-11ed-8c0a-1250e2f4dc61/kops delete cluster --name ha-migration.k8s.local --yes Error: error reading cluster configuration: Cluster.kops.k8s.io "ha-migration.k8s.local" not found I0202 22:51:01.235713 5790 gcs.go:51] gsutil ls -b -p k8s-jkns-gce-soak-1-2 gs://k8s-jkns-gce-soak-1-2-state-c3 I0202 22:51:01.235769 5790 local.go:42] ⚙️ gsutil ls -b -p k8s-jkns-gce-soak-1-2 gs://k8s-jkns-gce-soak-1-2-state-c3 I0202 22:51:03.482507 5790 gcs.go:70] gsutil mb -p k8s-jkns-gce-soak-1-2 gs://k8s-jkns-gce-soak-1-2-state-c3 I0202 22:51:03.482560 5790 local.go:42] ⚙️ gsutil mb -p k8s-jkns-gce-soak-1-2 gs://k8s-jkns-gce-soak-1-2-state-c3 Creating gs://k8s-jkns-gce-soak-1-2-state-c3/... I0202 22:51:06.248832 5790 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2023/02/02 22:51:06 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I0202 22:51:06.262087 5790 http.go:37] curl https://ip.jsb.workers.dev I0202 22:51:06.372186 5790 up.go:167] /home/prow/go/src/k8s.io/kops/_rundir/c38c579c-a34b-11ed-8c0a-1250e2f4dc61/kops create cluster --name ha-migration.k8s.local --cloud gce --kubernetes-version v1.23.16 --ssh-public-key /tmp/kops-ssh4289614941/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --node-count=1 --master-count=3 --zones=us-central1-a,us-central1-b,us-central1-c --master-zones=us-central1-a,us-central1-b,us-central1-c --gce-service-account=default --admin-access 35.224.141.93/32 --master-volume-size 48 --node-volume-size 48 --master-size e2-standard-2 --project k8s-jkns-gce-soak-1-2 I0202 22:51:06.372233 5790 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/c38c579c-a34b-11ed-8c0a-1250e2f4dc61/kops create cluster --name ha-migration.k8s.local --cloud gce --kubernetes-version v1.23.16 --ssh-public-key /tmp/kops-ssh4289614941/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --node-count=1 --master-count=3 --zones=us-central1-a,us-central1-b,us-central1-c --master-zones=us-central1-a,us-central1-b,us-central1-c --gce-service-account=default --admin-access 35.224.141.93/32 --master-volume-size 48 --node-volume-size 48 --master-size e2-standard-2 --project k8s-jkns-gce-soak-1-2 I0202 22:51:06.426211 6112 create_cluster.go:882] Using SSH public key: /tmp/kops-ssh4289614941/key.pub I0202 22:51:06.634327 6112 new_cluster.go:573] VMs will be configured to use specified Service Account: default I0202 22:51:06.634464 6112 new_cluster.go:1339] Cloud Provider ID: "gce" ... skipping 527 lines ... I0202 22:51:19.181766 6133 keypair.go:226] Issuing new certificate: "etcd-clients-ca" I0202 22:51:19.184202 6133 keypair.go:226] Issuing new certificate: "etcd-peers-ca-events" W0202 22:51:19.382617 6133 vfs_keystorereader.go:143] CA private key was not found I0202 22:51:19.480362 6133 keypair.go:226] Issuing new certificate: "service-account" I0202 22:51:19.570282 6133 keypair.go:226] Issuing new certificate: "kubernetes-ca" I0202 22:51:31.615080 6133 executor.go:111] Tasks: 50 done / 91 total; 23 can run W0202 22:51:44.502328 6133 executor.go:139] error running task "ForwardingRule/api-ha-migration-k8s-local" (9m47s remaining to succeed): error creating ForwardingRule "api-ha-migration-k8s-local": googleapi: Error 400: The resource 'projects/k8s-jkns-gce-soak-1-2/regions/us-central1/targetPools/api-ha-migration-k8s-local' is not ready, resourceNotReady I0202 22:51:44.502391 6133 executor.go:111] Tasks: 72 done / 91 total; 13 can run I0202 22:51:51.465188 6133 executor.go:111] Tasks: 85 done / 91 total; 6 can run I0202 22:52:05.496499 6133 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0202 22:52:05.571998 6133 update_cluster.go:323] Exporting kubeconfig for cluster kOps has set your kubectl context to ha-migration.k8s.local ... skipping 7 lines ... * read about installing addons at: https://kops.sigs.k8s.io/addons. I0202 22:52:15.926226 5790 up.go:251] /home/prow/go/src/k8s.io/kops/_rundir/c38c579c-a34b-11ed-8c0a-1250e2f4dc61/kops validate cluster --name ha-migration.k8s.local --count 10 --wait 15m0s I0202 22:52:15.926279 5790 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/c38c579c-a34b-11ed-8c0a-1250e2f4dc61/kops validate cluster --name ha-migration.k8s.local --count 10 --wait 15m0s Validating cluster ha-migration.k8s.local W0202 22:52:46.726716 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": dial tcp 35.225.87.32:443: i/o timeout W0202 22:53:03.886441 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": dial tcp 35.225.87.32:443: connect: connection refused W0202 22:53:13.897521 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": dial tcp 35.225.87.32:443: connect: connection refused W0202 22:53:23.904374 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": dial tcp 35.225.87.32:443: connect: connection refused W0202 22:53:33.914642 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": dial tcp 35.225.87.32:443: connect: connection refused W0202 22:53:43.919509 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": dial tcp 35.225.87.32:443: connect: connection refused W0202 22:53:53.928703 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": dial tcp 35.225.87.32:443: connect: connection refused W0202 22:54:03.935311 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": dial tcp 35.225.87.32:443: connect: connection refused W0202 22:54:13.940689 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": dial tcp 35.225.87.32:443: connect: connection refused W0202 22:54:23.944399 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": dial tcp 35.225.87.32:443: connect: connection refused W0202 22:54:33.947346 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": dial tcp 35.225.87.32:443: connect: connection refused W0202 22:54:53.962638 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": net/http: TLS handshake timeout W0202 22:55:13.970718 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": net/http: TLS handshake timeout W0202 22:55:31.722661 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": dial tcp 35.225.87.32:443: connect: connection refused - error from a previous attempt: read tcp 10.60.156.238:50082->35.225.87.32:443: read: connection reset by peer W0202 22:55:41.726377 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": dial tcp 35.225.87.32:443: connect: connection refused I0202 22:55:49.963595 5790 boskos.go:86] Sending heartbeat to Boskos W0202 22:56:01.734433 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": net/http: TLS handshake timeout W0202 22:56:13.931294 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": dial tcp 35.225.87.32:443: connect: connection refused - error from a previous attempt: read tcp 10.60.156.238:55326->35.225.87.32:443: read: connection reset by peer W0202 22:56:23.934113 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": dial tcp 35.225.87.32:443: connect: connection refused W0202 22:56:43.937597 6153 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.225.87.32/api/v1/nodes": net/http: TLS handshake timeout I0202 22:56:59.932413 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-c ControlPlane e2-standard-2 1 1 us-central1 ... skipping 8 lines ... KIND NAME MESSAGE Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-2/zones/us-central1-a/instances/control-plane-us-central1-a-0l8g machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-2/zones/us-central1-a/instances/control-plane-us-central1-a-0l8g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-2/zones/us-central1-a/instances/nodes-us-central1-a-4m34 machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-2/zones/us-central1-a/instances/nodes-us-central1-a-4m34" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-2/zones/us-central1-b/instances/control-plane-us-central1-b-njjf machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-2/zones/us-central1-b/instances/control-plane-us-central1-b-njjf" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-2/zones/us-central1-c/instances/control-plane-us-central1-c-klx4 machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-2/zones/us-central1-c/instances/control-plane-us-central1-c-klx4" has not yet joined cluster Validation Failed W0202 22:57:00.890905 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 22:57:11.323542 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 12 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-2/zones/us-central1-a/instances/control-plane-us-central1-a-0l8g machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-2/zones/us-central1-a/instances/control-plane-us-central1-a-0l8g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-2/zones/us-central1-a/instances/nodes-us-central1-a-4m34 machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-2/zones/us-central1-a/instances/nodes-us-central1-a-4m34" has not yet joined cluster Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/kube-controller-manager-control-plane-us-central1-b-njjf system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-b-njjf" is pending Validation Failed W0202 22:57:12.287568 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 22:57:22.770081 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 14 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/etcd-manager-events-control-plane-us-central1-b-njjf system-cluster-critical pod "etcd-manager-events-control-plane-us-central1-b-njjf" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-b-njjf system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-b-njjf" is not ready (kube-controller-manager) Pod kube-system/kube-scheduler-control-plane-us-central1-b-njjf system-cluster-critical pod "kube-scheduler-control-plane-us-central1-b-njjf" is pending Validation Failed W0202 22:57:23.687261 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 22:57:34.129324 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 13 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-2/zones/us-central1-a/instances/nodes-us-central1-a-4m34 machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-2/zones/us-central1-a/instances/nodes-us-central1-a-4m34" has not yet joined cluster Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/etcd-manager-main-control-plane-us-central1-b-njjf system-cluster-critical pod "etcd-manager-main-control-plane-us-central1-b-njjf" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-b-njjf system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-b-njjf" is not ready (kube-controller-manager) Validation Failed W0202 22:57:35.010542 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 22:57:45.462785 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 14 lines ... Node control-plane-us-central1-a-0l8g node "control-plane-us-central1-a-0l8g" of role "control-plane" is not ready Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/etcd-manager-events-control-plane-us-central1-c-klx4 system-cluster-critical pod "etcd-manager-events-control-plane-us-central1-c-klx4" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-b-njjf system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-b-njjf" is not ready (kube-controller-manager) Validation Failed W0202 22:57:46.235017 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 22:57:56.663377 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 22 lines ... Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Pod kube-system/kops-controller-87k6r system-cluster-critical pod "kops-controller-87k6r" is pending Pod kube-system/kops-controller-k5dhr system-cluster-critical pod "kops-controller-k5dhr" is pending Pod kube-system/kops-controller-r2482 system-cluster-critical pod "kops-controller-r2482" is pending Validation Failed W0202 22:57:57.662372 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 22:58:08.064399 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 21 lines ... Pod kube-system/cilium-p4lqb system-node-critical pod "cilium-p4lqb" is pending Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Pod kube-system/kops-controller-k5dhr system-cluster-critical pod "kops-controller-k5dhr" is pending Validation Failed W0202 22:58:08.923362 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 22:58:19.408031 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 18 lines ... Pod kube-system/cilium-fl9d5 system-node-critical pod "cilium-fl9d5" is pending Pod kube-system/cilium-p4lqb system-node-critical pod "cilium-p4lqb" is pending Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 22:58:20.308495 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 22:58:30.688901 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 19 lines ... Pod kube-system/cilium-p4lqb system-node-critical pod "cilium-p4lqb" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Pod kube-system/etcd-manager-main-control-plane-us-central1-c-klx4 system-cluster-critical pod "etcd-manager-main-control-plane-us-central1-c-klx4" is pending Validation Failed W0202 22:58:31.619203 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 22:58:42.046760 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/cilium-fl9d5 system-node-critical pod "cilium-fl9d5" is pending Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 22:58:42.861120 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 22:58:53.300524 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/cilium-fl9d5 system-node-critical pod "cilium-fl9d5" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 22:58:54.150017 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 22:59:04.622549 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 17 lines ... Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Pod kube-system/kube-apiserver-control-plane-us-central1-a-0l8g system-cluster-critical pod "kube-apiserver-control-plane-us-central1-a-0l8g" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-c-klx4 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-klx4" is pending Validation Failed W0202 22:59:05.456474 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 22:59:15.815640 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Pod kube-system/etcd-manager-main-control-plane-us-central1-a-0l8g system-cluster-critical pod "etcd-manager-main-control-plane-us-central1-a-0l8g" is pending Validation Failed W0202 22:59:16.652757 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 22:59:27.056757 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 22:59:27.895997 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 22:59:38.379292 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 22:59:39.253392 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 22:59:49.657446 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 22:59:50.572426 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:00:01.037383 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:00:01.909330 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:00:12.368590 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:00:13.238390 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:00:23.694868 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:00:24.543509 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:00:35.001329 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:00:35.872237 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:00:46.308793 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:00:47.234452 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:00:49.975982 5790 boskos.go:86] Sending heartbeat to Boskos I0202 23:00:57.670466 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:00:58.488800 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:01:08.892604 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:01:09.699494 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:01:20.120185 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:01:20.977561 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:01:31.483434 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:01:32.348901 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:01:42.799094 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:01:43.611284 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:01:54.096689 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:01:54.962333 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:02:05.375933 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:02:06.269309 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:02:16.717732 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:02:17.573529 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:02:28.036827 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:02:28.895918 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:02:39.339828 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:02:40.257078 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:02:50.753976 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:02:51.578467 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:03:02.114750 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:03:02.981864 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:03:13.402774 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:03:14.259626 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:03:24.723327 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:03:25.643651 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:03:36.135335 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:03:36.949316 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:03:47.378812 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:03:48.251060 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:03:58.627131 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:03:59.457475 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:04:09.854926 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:04:10.787008 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:04:21.196762 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:04:22.036992 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:04:32.499851 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:04:33.343134 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:04:43.744697 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:04:44.651004 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:04:55.105300 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:04:55.893190 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:05:06.276717 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:05:07.087694 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:05:17.516347 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:05:18.360524 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:05:28.750695 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:05:29.615310 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:05:40.067030 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:05:40.885923 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:05:49.988941 5790 boskos.go:86] Sending heartbeat to Boskos I0202 23:05:51.336421 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:05:52.180222 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:06:02.569632 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:06:03.443102 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:06:13.860210 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:06:14.743053 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:06:25.203835 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:06:26.057134 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:06:36.491008 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:06:37.338458 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:06:47.741217 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:06:48.597116 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:06:59.003591 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:06:59.912763 6153 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 23:07:10.375184 6153 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-njjf node "control-plane-us-central1-b-njjf" of role "control-plane" is not ready Node control-plane-us-central1-c-klx4 node "control-plane-us-central1-c-klx4" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-vvxzf system-cluster-critical pod "coredns-5d479c4f64-vvxzf" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gxkr7 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gxkr7" is pending Pod kube-system/dns-controller-6f69f47785-bnqpb system-cluster-critical pod "dns-controller-6f69f47785-bnqpb" is pending Validation Failed W0202 23:07:11.205447 6153 validate_cluster.go:232] (will retry): cluster not yet healthy Error: validation failed: wait time exceeded during validation I0202 23:07:21.217239 5790 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/c38c579c-a34b-11ed-8c0a-1250e2f4dc61/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh4289614941/key --ssh-user prow I0202 23:07:21.217350 5790 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/c38c579c-a34b-11ed-8c0a-1250e2f4dc61/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh4289614941/key --ssh-user prow I0202 23:07:21.693930 6163 gce.go:99] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] 2023/02/02 23:07:26 Dumping node control-plane-us-central1-a-0l8g 2023/02/02 23:07:30 error dumping node control-plane-us-central1-a-0l8g: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/02/02 23:07:30 error dumping node control-plane-us-central1-a-0l8g: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/02/02 23:07:30 Dumping node control-plane-us-central1-b-njjf 2023/02/02 23:07:33 error dumping node control-plane-us-central1-b-njjf: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/02/02 23:07:33 error dumping node control-plane-us-central1-b-njjf: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/02/02 23:07:33 Dumping node control-plane-us-central1-c-klx4 2023/02/02 23:07:36 error dumping node control-plane-us-central1-c-klx4: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/02/02 23:07:36 error dumping node control-plane-us-central1-c-klx4: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/02/02 23:07:36 dumping node not registered in kubernetes: 35.239.148.210 2023/02/02 23:07:36 Dumping node 35.239.148.210 I0202 23:07:38.618602 5790 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/c38c579c-a34b-11ed-8c0a-1250e2f4dc61/kops get cluster --name ha-migration.k8s.local -o yaml I0202 23:07:38.618641 5790 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/c38c579c-a34b-11ed-8c0a-1250e2f4dc61/kops get cluster --name ha-migration.k8s.local -o yaml I0202 23:07:38.794241 5790 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/c38c579c-a34b-11ed-8c0a-1250e2f4dc61/kops get instancegroups --name ha-migration.k8s.local -o yaml I0202 23:07:38.794284 5790 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/c38c579c-a34b-11ed-8c0a-1250e2f4dc61/kops get instancegroups --name ha-migration.k8s.local -o yaml ... skipping 167 lines ... Deleted cluster: "ha-migration.k8s.local" I0202 23:10:00.627914 5790 gcs.go:88] gsutil -u k8s-jkns-gce-soak-1-2 rm -r gs://k8s-jkns-gce-soak-1-2-state-c3 I0202 23:10:00.627995 5790 local.go:42] ⚙️ gsutil -u k8s-jkns-gce-soak-1-2 rm -r gs://k8s-jkns-gce-soak-1-2-state-c3 Removing gs://k8s-jkns-gce-soak-1-2-state-c3/... I0202 23:10:03.697705 5790 down.go:62] releasing boskos project I0202 23:10:03.707610 5790 boskos.go:83] Boskos heartbeat func received signal to close Error: exit status 1 + EXIT_VALUE=1 + set +o xtrace