e2e-kops-gce-leader-migration #1621007716250030080
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 1 failed / 1 succeeded |
| Started | |
| Elapsed | 20m21s |
| Revision | master |
exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest2 Down
... skipping 156 lines ... .............................Updated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-etcd2-1-6]. ..done. WARNING: No host aliases were added to your SSH configs because you do not have any running instances. Try running this command again after running some instances. I0202 04:50:52.983941 5743 up.go:44] Cleaning up any leaked resources from previous cluster I0202 04:50:52.984084 5743 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/e35c6620-a2b4-11ed-8c0a-1250e2f4dc61/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1577918993/key --ssh-user prow I0202 04:50:52.984102 5743 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/e35c6620-a2b4-11ed-8c0a-1250e2f4dc61/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1577918993/key --ssh-user prow Error: Cluster.kops.k8s.io "ha-migration.k8s.local" not found W0202 04:50:53.211727 5743 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0202 04:50:53.211832 5743 down.go:48] /home/prow/go/src/k8s.io/kops/_rundir/e35c6620-a2b4-11ed-8c0a-1250e2f4dc61/kops delete cluster --name ha-migration.k8s.local --yes I0202 04:50:53.211851 5743 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/e35c6620-a2b4-11ed-8c0a-1250e2f4dc61/kops delete cluster --name ha-migration.k8s.local --yes Error: error reading cluster configuration: Cluster.kops.k8s.io "ha-migration.k8s.local" not found I0202 04:50:53.407093 5743 gcs.go:51] gsutil ls -b -p k8s-jkns-gce-etcd2-1-6 gs://k8s-jkns-gce-etcd2-1-6-state-e3 I0202 04:50:53.407139 5743 local.go:42] ⚙️ gsutil ls -b -p k8s-jkns-gce-etcd2-1-6 gs://k8s-jkns-gce-etcd2-1-6-state-e3 I0202 04:50:56.135949 5743 gcs.go:70] gsutil mb -p k8s-jkns-gce-etcd2-1-6 gs://k8s-jkns-gce-etcd2-1-6-state-e3 I0202 04:50:56.135999 5743 local.go:42] ⚙️ gsutil mb -p k8s-jkns-gce-etcd2-1-6 gs://k8s-jkns-gce-etcd2-1-6-state-e3 Creating gs://k8s-jkns-gce-etcd2-1-6-state-e3/... I0202 04:50:58.640702 5743 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2023/02/02 04:50:58 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I0202 04:50:58.654154 5743 http.go:37] curl https://ip.jsb.workers.dev I0202 04:50:58.751878 5743 up.go:167] /home/prow/go/src/k8s.io/kops/_rundir/e35c6620-a2b4-11ed-8c0a-1250e2f4dc61/kops create cluster --name ha-migration.k8s.local --cloud gce --kubernetes-version v1.23.16 --ssh-public-key /tmp/kops-ssh1577918993/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --node-count=1 --master-count=3 --zones=us-central1-a,us-central1-b,us-central1-c --master-zones=us-central1-a,us-central1-b,us-central1-c --gce-service-account=default --admin-access 35.202.151.134/32 --master-volume-size 48 --node-volume-size 48 --master-size e2-standard-2 --project k8s-jkns-gce-etcd2-1-6 I0202 04:50:58.752020 5743 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/e35c6620-a2b4-11ed-8c0a-1250e2f4dc61/kops create cluster --name ha-migration.k8s.local --cloud gce --kubernetes-version v1.23.16 --ssh-public-key /tmp/kops-ssh1577918993/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --node-count=1 --master-count=3 --zones=us-central1-a,us-central1-b,us-central1-c --master-zones=us-central1-a,us-central1-b,us-central1-c --gce-service-account=default --admin-access 35.202.151.134/32 --master-volume-size 48 --node-volume-size 48 --master-size e2-standard-2 --project k8s-jkns-gce-etcd2-1-6 I0202 04:50:58.804889 6064 create_cluster.go:882] Using SSH public key: /tmp/kops-ssh1577918993/key.pub I0202 04:50:59.049610 6064 new_cluster.go:573] VMs will be configured to use specified Service Account: default I0202 04:50:59.049742 6064 new_cluster.go:1339] Cloud Provider ID: "gce" ... skipping 527 lines ... I0202 04:51:11.076158 6083 keypair.go:226] Issuing new certificate: "etcd-peers-ca-main" I0202 04:51:11.077818 6083 keypair.go:226] Issuing new certificate: "etcd-manager-ca-events" W0202 04:51:11.176072 6083 vfs_keystorereader.go:143] CA private key was not found I0202 04:51:11.277040 6083 keypair.go:226] Issuing new certificate: "kubernetes-ca" I0202 04:51:11.487694 6083 keypair.go:226] Issuing new certificate: "service-account" I0202 04:51:23.729589 6083 executor.go:111] Tasks: 50 done / 91 total; 23 can run W0202 04:51:33.809228 6083 executor.go:139] error running task "ForwardingRule/api-ha-migration-k8s-local" (9m49s remaining to succeed): error creating ForwardingRule "api-ha-migration-k8s-local": googleapi: Error 400: The resource 'projects/k8s-jkns-gce-etcd2-1-6/regions/us-central1/targetPools/api-ha-migration-k8s-local' is not ready, resourceNotReady I0202 04:51:33.809292 6083 executor.go:111] Tasks: 72 done / 91 total; 13 can run I0202 04:51:55.591510 6083 executor.go:111] Tasks: 85 done / 91 total; 6 can run I0202 04:52:09.445944 6083 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0202 04:52:09.613275 6083 update_cluster.go:323] Exporting kubeconfig for cluster kOps has set your kubectl context to ha-migration.k8s.local ... skipping 7 lines ... * read about installing addons at: https://kops.sigs.k8s.io/addons. I0202 04:52:20.000156 5743 up.go:251] /home/prow/go/src/k8s.io/kops/_rundir/e35c6620-a2b4-11ed-8c0a-1250e2f4dc61/kops validate cluster --name ha-migration.k8s.local --count 10 --wait 15m0s I0202 04:52:20.000214 5743 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/e35c6620-a2b4-11ed-8c0a-1250e2f4dc61/kops validate cluster --name ha-migration.k8s.local --count 10 --wait 15m0s Validating cluster ha-migration.k8s.local W0202 04:52:50.571878 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: i/o timeout W0202 04:53:16.061511 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:53:26.066747 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:53:36.069519 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:53:46.073259 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:53:56.076800 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:54:06.079400 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:54:16.082232 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:54:26.085693 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:54:36.089832 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:54:46.098501 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:54:56.106474 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:55:16.121996 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": net/http: TLS handshake timeout W0202 04:55:36.136120 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": net/http: TLS handshake timeout I0202 04:55:42.737690 5743 boskos.go:86] Sending heartbeat to Boskos W0202 04:55:46.142952 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:55:56.152180 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:56:16.160806 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": net/http: TLS handshake timeout W0202 04:56:26.164314 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:56:36.172220 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:56:46.180148 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": dial tcp 34.172.144.223:443: connect: connection refused W0202 04:57:06.189231 6103 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.172.144.223/api/v1/nodes": net/http: TLS handshake timeout I0202 04:57:16.694107 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-c ControlPlane e2-standard-2 1 1 us-central1 ... skipping 10 lines ... KIND NAME MESSAGE Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-etcd2-1-6/zones/us-central1-a/instances/control-plane-us-central1-a-52bt machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-etcd2-1-6/zones/us-central1-a/instances/control-plane-us-central1-a-52bt" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-etcd2-1-6/zones/us-central1-a/instances/nodes-us-central1-a-3xgz machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-etcd2-1-6/zones/us-central1-a/instances/nodes-us-central1-a-3xgz" has not yet joined cluster Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Validation Failed W0202 04:57:17.620567 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 04:57:28.011224 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/kube-controller-manager-control-plane-us-central1-a-52bt system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-a-52bt" is not ready (kube-controller-manager) Pod kube-system/kube-controller-manager-control-plane-us-central1-c-0zs0 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-0zs0" is pending Pod kube-system/kube-scheduler-control-plane-us-central1-c-0zs0 system-cluster-critical pod "kube-scheduler-control-plane-us-central1-c-0zs0" is pending Validation Failed W0202 04:57:28.999736 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 04:57:39.400615 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/etcd-manager-main-control-plane-us-central1-b-3hzj system-cluster-critical pod "etcd-manager-main-control-plane-us-central1-b-3hzj" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-a-52bt system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-a-52bt" is not ready (kube-controller-manager) Pod kube-system/kube-controller-manager-control-plane-us-central1-b-3hzj system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-b-3hzj" is not ready (kube-controller-manager) Pod kube-system/kube-controller-manager-control-plane-us-central1-c-0zs0 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-0zs0" is not ready (kube-controller-manager) Validation Failed W0202 04:57:40.287169 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 04:57:50.703940 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/etcd-manager-events-control-plane-us-central1-a-52bt system-cluster-critical pod "etcd-manager-events-control-plane-us-central1-a-52bt" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-a-52bt system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-a-52bt" is not ready (kube-controller-manager) Pod kube-system/kube-controller-manager-control-plane-us-central1-b-3hzj system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-b-3hzj" is not ready (kube-controller-manager) Pod kube-system/kube-controller-manager-control-plane-us-central1-c-0zs0 system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-0zs0" is not ready (kube-controller-manager) Validation Failed W0202 04:57:51.508845 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 04:58:01.889380 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/etcd-manager-events-control-plane-us-central1-c-0zs0 system-cluster-critical pod "etcd-manager-events-control-plane-us-central1-c-0zs0" is pending Pod kube-system/kube-apiserver-control-plane-us-central1-a-52bt system-cluster-critical pod "kube-apiserver-control-plane-us-central1-a-52bt" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-a-52bt system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-a-52bt" is not ready (kube-controller-manager) Validation Failed W0202 04:58:02.840606 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 04:58:13.303798 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 21 lines ... Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Pod kube-system/kube-apiserver-control-plane-us-central1-b-3hzj system-cluster-critical pod "kube-apiserver-control-plane-us-central1-b-3hzj" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-a-52bt system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-a-52bt" is not ready (kube-controller-manager) Validation Failed W0202 04:58:14.220228 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 04:58:24.661933 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 21 lines ... Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Pod kube-system/kube-apiserver-control-plane-us-central1-c-0zs0 system-cluster-critical pod "kube-apiserver-control-plane-us-central1-c-0zs0" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-a-52bt system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-a-52bt" is not ready (kube-controller-manager) Validation Failed W0202 04:58:25.514982 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 04:58:35.997732 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 19 lines ... Pod kube-system/cilium-pd8cn system-node-critical pod "cilium-pd8cn" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-a-52bt system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-a-52bt" is not ready (kube-controller-manager) Validation Failed W0202 04:58:36.846322 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 04:58:47.266798 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 17 lines ... Pod kube-system/cilium-bjw4x system-node-critical pod "cilium-bjw4x" is not ready (cilium-agent) Pod kube-system/cilium-lzfm9 system-node-critical pod "cilium-lzfm9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 04:58:48.093825 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 04:58:58.589305 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/cilium-bjw4x system-node-critical pod "cilium-bjw4x" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 04:58:59.410362 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 04:59:09.884516 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 04:59:10.734910 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 04:59:21.145795 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 04:59:22.027101 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 04:59:32.422573 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 04:59:33.331439 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 04:59:43.825945 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 04:59:44.655115 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 04:59:55.030095 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 04:59:55.854949 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:00:06.310754 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:00:07.153544 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:00:17.564057 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:00:18.439700 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:00:28.837797 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:00:29.666497 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:00:40.113335 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:00:40.923984 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:00:42.747157 5743 boskos.go:86] Sending heartbeat to Boskos I0202 05:00:51.327652 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:00:52.148340 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:01:02.529282 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:01:03.404949 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:01:13.810112 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:01:14.672716 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:01:25.069564 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:01:25.958198 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:01:36.387911 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:01:37.216686 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:01:47.640861 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:01:48.467578 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:01:58.898086 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:01:59.729153 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:02:10.199467 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:02:11.061752 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:02:21.409780 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:02:22.255826 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:02:32.655945 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:02:33.588799 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:02:44.033327 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:02:44.907419 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:02:55.288310 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:02:56.111400 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:03:06.528895 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:03:07.388609 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:03:17.834328 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:03:18.709174 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:03:29.154729 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:03:29.987593 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:03:40.459043 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:03:41.307365 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:03:51.714149 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:03:52.609127 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:04:03.051460 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:04:03.976577 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:04:14.410820 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:04:15.268503 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:04:25.665839 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:04:26.495199 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:04:36.907509 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:04:37.718162 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:04:48.141209 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:04:48.928075 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:04:59.422233 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:05:00.216366 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:05:10.621143 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:05:11.444605 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:05:21.932650 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:05:22.794016 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:05:33.216313 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:05:34.114931 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:05:42.758193 5743 boskos.go:86] Sending heartbeat to Boskos I0202 05:05:44.496274 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:05:45.350259 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:05:55.776137 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:05:56.558807 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:06:06.953183 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:06:07.849550 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:06:18.265384 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:06:19.070104 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:06:29.520522 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:06:30.381062 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:06:40.830190 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:06:41.746901 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:06:52.164142 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:06:53.034401 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:07:03.410416 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:07:04.228236 6103 validate_cluster.go:232] (will retry): cluster not yet healthy I0202 05:07:14.669790 6103 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-3hzj node "control-plane-us-central1-b-3hzj" of role "control-plane" is not ready Node control-plane-us-central1-c-0zs0 node "control-plane-us-central1-c-0zs0" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-mrggv system-cluster-critical pod "coredns-5d479c4f64-mrggv" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-p5kg8 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-p5kg8" is pending Pod kube-system/dns-controller-6f69f47785-n6ksn system-cluster-critical pod "dns-controller-6f69f47785-n6ksn" is pending Validation Failed W0202 05:07:15.649839 6103 validate_cluster.go:232] (will retry): cluster not yet healthy Error: validation failed: wait time exceeded during validation I0202 05:07:25.661776 5743 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/e35c6620-a2b4-11ed-8c0a-1250e2f4dc61/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1577918993/key --ssh-user prow I0202 05:07:25.661818 5743 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/e35c6620-a2b4-11ed-8c0a-1250e2f4dc61/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1577918993/key --ssh-user prow I0202 05:07:26.088103 6113 gce.go:99] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] 2023/02/02 05:07:30 Dumping node control-plane-us-central1-a-52bt 2023/02/02 05:07:33 error dumping node control-plane-us-central1-a-52bt: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/02/02 05:07:33 error dumping node control-plane-us-central1-a-52bt: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/02/02 05:07:33 Dumping node control-plane-us-central1-b-3hzj 2023/02/02 05:07:38 error dumping node control-plane-us-central1-b-3hzj: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/02/02 05:07:38 error dumping node control-plane-us-central1-b-3hzj: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/02/02 05:07:38 Dumping node control-plane-us-central1-c-0zs0 2023/02/02 05:07:44 error dumping node control-plane-us-central1-c-0zs0: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/02/02 05:07:44 error dumping node control-plane-us-central1-c-0zs0: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/02/02 05:07:44 dumping node not registered in kubernetes: 35.232.8.34 2023/02/02 05:07:44 Dumping node 35.232.8.34 I0202 05:07:46.008091 5743 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/e35c6620-a2b4-11ed-8c0a-1250e2f4dc61/kops get cluster --name ha-migration.k8s.local -o yaml I0202 05:07:46.008137 5743 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/e35c6620-a2b4-11ed-8c0a-1250e2f4dc61/kops get cluster --name ha-migration.k8s.local -o yaml I0202 05:07:46.146551 5743 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/e35c6620-a2b4-11ed-8c0a-1250e2f4dc61/kops get instancegroups --name ha-migration.k8s.local -o yaml I0202 05:07:46.146590 5743 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/e35c6620-a2b4-11ed-8c0a-1250e2f4dc61/kops get instancegroups --name ha-migration.k8s.local -o yaml ... skipping 167 lines ... Deleted cluster: "ha-migration.k8s.local" I0202 05:09:54.373400 5743 gcs.go:88] gsutil -u k8s-jkns-gce-etcd2-1-6 rm -r gs://k8s-jkns-gce-etcd2-1-6-state-e3 I0202 05:09:54.373445 5743 local.go:42] ⚙️ gsutil -u k8s-jkns-gce-etcd2-1-6 rm -r gs://k8s-jkns-gce-etcd2-1-6-state-e3 Removing gs://k8s-jkns-gce-etcd2-1-6-state-e3/... I0202 05:09:57.043084 5743 down.go:62] releasing boskos project I0202 05:09:57.051317 5743 boskos.go:83] Boskos heartbeat func received signal to close Error: exit status 1 + EXIT_VALUE=1 + set +o xtrace