e2e-kops-gce-leader-migration #1620735876457828352
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 1 failed / 1 succeeded |
| Started | |
| Elapsed | 20m7s |
| Revision | master |
exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest2 Down
... skipping 156 lines ... ...................................Updated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4]. .done. WARNING: No host aliases were added to your SSH configs because you do not have any running instances. Try running this command again after running some instances. I0201 10:50:31.889401 5758 up.go:44] Cleaning up any leaked resources from previous cluster I0201 10:50:31.889529 5758 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/fc9a7cbb-a21d-11ed-9efc-5e7820dba16b/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh415695073/key --ssh-user prow I0201 10:50:31.889545 5758 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/fc9a7cbb-a21d-11ed-9efc-5e7820dba16b/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh415695073/key --ssh-user prow Error: Cluster.kops.k8s.io "ha-migration.k8s.local" not found W0201 10:50:32.240133 5758 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0201 10:50:32.240186 5758 down.go:48] /home/prow/go/src/k8s.io/kops/_rundir/fc9a7cbb-a21d-11ed-9efc-5e7820dba16b/kops delete cluster --name ha-migration.k8s.local --yes I0201 10:50:32.240197 5758 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/fc9a7cbb-a21d-11ed-9efc-5e7820dba16b/kops delete cluster --name ha-migration.k8s.local --yes Error: error reading cluster configuration: Cluster.kops.k8s.io "ha-migration.k8s.local" not found I0201 10:50:32.499062 5758 gcs.go:51] gsutil ls -b -p k8s-jkns-gce-soak-1-4 gs://k8s-jkns-gce-soak-1-4-state-fc I0201 10:50:32.499093 5758 local.go:42] ⚙️ gsutil ls -b -p k8s-jkns-gce-soak-1-4 gs://k8s-jkns-gce-soak-1-4-state-fc I0201 10:50:34.479994 5758 gcs.go:70] gsutil mb -p k8s-jkns-gce-soak-1-4 gs://k8s-jkns-gce-soak-1-4-state-fc I0201 10:50:34.480033 5758 local.go:42] ⚙️ gsutil mb -p k8s-jkns-gce-soak-1-4 gs://k8s-jkns-gce-soak-1-4-state-fc Creating gs://k8s-jkns-gce-soak-1-4-state-fc/... I0201 10:50:36.997139 5758 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2023/02/01 10:50:37 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I0201 10:50:37.011455 5758 http.go:37] curl https://ip.jsb.workers.dev I0201 10:50:37.122970 5758 up.go:167] /home/prow/go/src/k8s.io/kops/_rundir/fc9a7cbb-a21d-11ed-9efc-5e7820dba16b/kops create cluster --name ha-migration.k8s.local --cloud gce --kubernetes-version v1.23.16 --ssh-public-key /tmp/kops-ssh415695073/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --node-count=1 --master-count=3 --zones=us-central1-a,us-central1-b,us-central1-c --master-zones=us-central1-a,us-central1-b,us-central1-c --gce-service-account=default --admin-access 35.238.154.178/32 --master-volume-size 48 --node-volume-size 48 --master-size e2-standard-2 --project k8s-jkns-gce-soak-1-4 I0201 10:50:37.123003 5758 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/fc9a7cbb-a21d-11ed-9efc-5e7820dba16b/kops create cluster --name ha-migration.k8s.local --cloud gce --kubernetes-version v1.23.16 --ssh-public-key /tmp/kops-ssh415695073/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --node-count=1 --master-count=3 --zones=us-central1-a,us-central1-b,us-central1-c --master-zones=us-central1-a,us-central1-b,us-central1-c --gce-service-account=default --admin-access 35.238.154.178/32 --master-volume-size 48 --node-volume-size 48 --master-size e2-standard-2 --project k8s-jkns-gce-soak-1-4 I0201 10:50:37.170706 6078 create_cluster.go:882] Using SSH public key: /tmp/kops-ssh415695073/key.pub I0201 10:50:37.436534 6078 new_cluster.go:573] VMs will be configured to use specified Service Account: default I0201 10:50:37.436735 6078 new_cluster.go:1339] Cloud Provider ID: "gce" ... skipping 527 lines ... I0201 10:50:49.491630 6095 keypair.go:226] Issuing new certificate: "etcd-clients-ca" I0201 10:50:49.491675 6095 keypair.go:226] Issuing new certificate: "etcd-peers-ca-events" W0201 10:50:49.674254 6095 vfs_keystorereader.go:143] CA private key was not found I0201 10:50:49.814732 6095 keypair.go:226] Issuing new certificate: "kubernetes-ca" I0201 10:50:49.867652 6095 keypair.go:226] Issuing new certificate: "service-account" I0201 10:51:01.873192 6095 executor.go:111] Tasks: 50 done / 91 total; 23 can run W0201 10:51:11.647840 6095 executor.go:139] error running task "ForwardingRule/api-ha-migration-k8s-local" (9m50s remaining to succeed): error creating ForwardingRule "api-ha-migration-k8s-local": googleapi: Error 400: The resource 'projects/k8s-jkns-gce-soak-1-4/regions/us-central1/targetPools/api-ha-migration-k8s-local' is not ready, resourceNotReady I0201 10:51:11.647909 6095 executor.go:111] Tasks: 72 done / 91 total; 13 can run I0201 10:51:18.395548 6095 executor.go:111] Tasks: 85 done / 91 total; 6 can run I0201 10:51:32.602889 6095 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0201 10:51:32.667031 6095 update_cluster.go:323] Exporting kubeconfig for cluster kOps has set your kubectl context to ha-migration.k8s.local ... skipping 7 lines ... * read about installing addons at: https://kops.sigs.k8s.io/addons. I0201 10:51:42.957176 5758 up.go:251] /home/prow/go/src/k8s.io/kops/_rundir/fc9a7cbb-a21d-11ed-9efc-5e7820dba16b/kops validate cluster --name ha-migration.k8s.local --count 10 --wait 15m0s I0201 10:51:42.957228 5758 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/fc9a7cbb-a21d-11ed-9efc-5e7820dba16b/kops validate cluster --name ha-migration.k8s.local --count 10 --wait 15m0s Validating cluster ha-migration.k8s.local W0201 10:52:13.561598 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: i/o timeout W0201 10:52:39.009516 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:52:49.015176 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:52:59.022724 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:53:09.030780 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:53:19.038474 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:53:29.041144 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:53:39.045606 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:53:49.048888 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:53:59.057590 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:54:09.065427 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:54:19.074011 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:54:39.088285 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": net/http: TLS handshake timeout W0201 10:54:49.095750 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:55:07.565367 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused - error from a previous attempt: read tcp 10.60.31.134:33284->34.135.188.195:443: read: connection reset by peer W0201 10:55:17.567852 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused I0201 10:55:21.363011 5758 boskos.go:86] Sending heartbeat to Boskos W0201 10:55:27.574354 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:55:37.580766 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:55:47.587042 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:55:57.590281 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": dial tcp 34.135.188.195:443: connect: connection refused W0201 10:56:17.597753 6116 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.135.188.195/api/v1/nodes": net/http: TLS handshake timeout I0201 10:56:28.032455 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-c ControlPlane e2-standard-2 1 1 us-central1 ... skipping 8 lines ... KIND NAME MESSAGE Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/control-plane-us-central1-a-p37f machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/control-plane-us-central1-a-p37f" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-b/instances/control-plane-us-central1-b-4jdm machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-b/instances/control-plane-us-central1-b-4jdm" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-c/instances/control-plane-us-central1-c-zvnk machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-c/instances/control-plane-us-central1-c-zvnk" has not yet joined cluster Validation Failed W0201 10:56:28.990052 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:56:39.418081 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 10 lines ... KIND NAME MESSAGE Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/control-plane-us-central1-a-p37f machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/control-plane-us-central1-a-p37f" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-b/instances/control-plane-us-central1-b-4jdm machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-b/instances/control-plane-us-central1-b-4jdm" has not yet joined cluster Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Validation Failed W0201 10:56:40.390540 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:56:50.768922 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 10 lines ... KIND NAME MESSAGE Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/control-plane-us-central1-a-p37f machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/control-plane-us-central1-a-p37f" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-b/instances/control-plane-us-central1-b-4jdm machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-b/instances/control-plane-us-central1-b-4jdm" has not yet joined cluster Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Validation Failed W0201 10:56:51.636019 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:57:02.018835 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 10 lines ... KIND NAME MESSAGE Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/control-plane-us-central1-a-p37f machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/control-plane-us-central1-a-p37f" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-b/instances/control-plane-us-central1-b-4jdm machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-b/instances/control-plane-us-central1-b-4jdm" has not yet joined cluster Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Validation Failed W0201 10:57:02.921168 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:57:13.338670 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 11 lines ... KIND NAME MESSAGE Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/control-plane-us-central1-a-p37f machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/control-plane-us-central1-a-p37f" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn" has not yet joined cluster Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Validation Failed W0201 10:57:14.438281 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:57:24.738416 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 13 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn" has not yet joined cluster Node control-plane-us-central1-a-p37f node "control-plane-us-central1-a-p37f" of role "control-plane" is not ready Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/kube-controller-manager-control-plane-us-central1-c-zvnk system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-zvnk" is not ready (kube-controller-manager) Validation Failed W0201 10:57:25.538511 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:57:35.931523 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 13 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn" has not yet joined cluster Node control-plane-us-central1-a-p37f node "control-plane-us-central1-a-p37f" of role "control-plane" is not ready Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/kube-controller-manager-control-plane-us-central1-c-zvnk system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-zvnk" is not ready (kube-controller-manager) Validation Failed W0201 10:57:36.830855 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:57:47.208836 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 13 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn" has not yet joined cluster Node control-plane-us-central1-a-p37f node "control-plane-us-central1-a-p37f" of role "control-plane" is not ready Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/kube-controller-manager-control-plane-us-central1-c-zvnk system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-c-zvnk" is not ready (kube-controller-manager) Validation Failed W0201 10:57:48.034554 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:57:58.364534 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 13 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-soak-1-4/zones/us-central1-a/instances/nodes-us-central1-a-bvsn" has not yet joined cluster Node control-plane-us-central1-a-p37f node "control-plane-us-central1-a-p37f" of role "control-plane" is not ready Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/etcd-manager-events-control-plane-us-central1-c-zvnk system-cluster-critical pod "etcd-manager-events-control-plane-us-central1-c-zvnk" is pending Validation Failed W0201 10:58:00.367665 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:58:10.810625 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 20 lines ... Pod kube-system/cilium-vfhzq system-node-critical pod "cilium-vfhzq" is pending Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Pod kube-system/etcd-manager-main-control-plane-us-central1-a-p37f system-cluster-critical pod "etcd-manager-main-control-plane-us-central1-a-p37f" is pending Validation Failed W0201 10:58:11.735008 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:58:22.200684 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 20 lines ... Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Pod kube-system/etcd-manager-main-control-plane-us-central1-b-4jdm system-cluster-critical pod "etcd-manager-main-control-plane-us-central1-b-4jdm" is pending Pod kube-system/kube-apiserver-control-plane-us-central1-b-4jdm system-cluster-critical pod "kube-apiserver-control-plane-us-central1-b-4jdm" is pending Validation Failed W0201 10:58:23.116340 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:58:33.466535 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 19 lines ... Pod kube-system/cilium-vfhzq system-node-critical pod "cilium-vfhzq" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Pod kube-system/kube-scheduler-control-plane-us-central1-b-4jdm system-cluster-critical pod "kube-scheduler-control-plane-us-central1-b-4jdm" is pending Validation Failed W0201 10:58:34.324106 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:58:44.677850 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 10:58:45.517438 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:58:55.841227 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 10:58:56.696003 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:59:07.017640 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 10:59:07.894565 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:59:18.287858 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 17 lines ... Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Pod kube-system/kube-controller-manager-control-plane-us-central1-b-4jdm system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-b-4jdm" is pending Validation Failed W0201 10:59:19.140916 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:59:29.470545 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 10:59:30.407417 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:59:40.826148 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 10:59:41.714838 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 10:59:52.080084 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 10:59:52.952874 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:00:03.456056 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:00:04.254361 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:00:14.640123 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:00:15.677949 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:00:21.373834 5758 boskos.go:86] Sending heartbeat to Boskos I0201 11:00:26.033928 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 17 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:00:26.877232 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:00:37.194310 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:00:38.180055 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:00:48.520807 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:00:49.423727 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:00:59.861777 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:01:00.692483 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:01:11.005487 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:01:11.937572 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:01:22.417326 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:01:23.373325 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:01:33.743580 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:01:34.627923 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:01:45.027910 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:01:45.907272 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:01:56.319118 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:01:57.104763 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:02:07.440298 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:02:08.314065 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:02:18.657960 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/cilium-cztv9 system-node-critical pod "cilium-cztv9" is not ready (cilium-agent) Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:02:19.530270 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:02:29.930903 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:02:30.839239 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:02:41.244303 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:02:42.154792 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:02:52.547054 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:02:53.377221 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:03:03.709482 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:03:04.561358 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:03:14.891703 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:03:15.743381 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:03:26.172803 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:03:27.002460 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:03:37.508215 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:03:38.319377 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:03:48.619861 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:03:49.512975 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:03:59.907259 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:04:00.820476 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:04:11.241514 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:04:12.048728 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:04:22.454975 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:04:23.295612 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:04:33.702210 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:04:34.585306 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:04:44.939068 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:04:45.760563 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:04:56.103217 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:04:56.946527 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:05:07.293587 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:05:08.197120 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:05:18.565075 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:05:19.640156 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:05:21.385543 5758 boskos.go:86] Sending heartbeat to Boskos I0201 11:05:30.059614 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 16 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:05:30.952989 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:05:41.299641 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:05:42.112555 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:05:52.518631 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:05:53.423558 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:06:03.847711 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:06:04.730871 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:06:15.080916 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:06:15.946854 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:06:26.342557 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:06:27.181411 6116 validate_cluster.go:232] (will retry): cluster not yet healthy I0201 11:06:37.542521 6116 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 control-plane-us-central1-b ControlPlane e2-standard-2 1 1 us-central1 ... skipping 15 lines ... Node control-plane-us-central1-b-4jdm node "control-plane-us-central1-b-4jdm" of role "control-plane" is not ready Node control-plane-us-central1-c-zvnk node "control-plane-us-central1-c-zvnk" of role "control-plane" is not ready Pod kube-system/coredns-5d479c4f64-qpkdd system-cluster-critical pod "coredns-5d479c4f64-qpkdd" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gb6mh system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gb6mh" is pending Pod kube-system/dns-controller-6f69f47785-jbmmt system-cluster-critical pod "dns-controller-6f69f47785-jbmmt" is pending Validation Failed W0201 11:06:38.348672 6116 validate_cluster.go:232] (will retry): cluster not yet healthy Error: validation failed: wait time exceeded during validation I0201 11:06:48.357957 5758 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/fc9a7cbb-a21d-11ed-9efc-5e7820dba16b/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh415695073/key --ssh-user prow I0201 11:06:48.358005 5758 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/fc9a7cbb-a21d-11ed-9efc-5e7820dba16b/kops toolbox dump --name ha-migration.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh415695073/key --ssh-user prow I0201 11:06:48.769840 6126 gce.go:99] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] 2023/02/01 11:06:53 Dumping node control-plane-us-central1-a-p37f 2023/02/01 11:06:57 error dumping node control-plane-us-central1-a-p37f: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/02/01 11:06:57 error dumping node control-plane-us-central1-a-p37f: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/02/01 11:06:57 Dumping node control-plane-us-central1-b-4jdm 2023/02/01 11:07:02 error dumping node control-plane-us-central1-b-4jdm: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/02/01 11:07:02 error dumping node control-plane-us-central1-b-4jdm: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/02/01 11:07:02 Dumping node control-plane-us-central1-c-zvnk 2023/02/01 11:07:05 error dumping node control-plane-us-central1-c-zvnk: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/02/01 11:07:05 error dumping node control-plane-us-central1-c-zvnk: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/02/01 11:07:05 dumping node not registered in kubernetes: 35.226.84.86 2023/02/01 11:07:05 Dumping node 35.226.84.86 I0201 11:07:07.432416 5758 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/fc9a7cbb-a21d-11ed-9efc-5e7820dba16b/kops get cluster --name ha-migration.k8s.local -o yaml I0201 11:07:07.432453 5758 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/fc9a7cbb-a21d-11ed-9efc-5e7820dba16b/kops get cluster --name ha-migration.k8s.local -o yaml I0201 11:07:07.582089 5758 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/fc9a7cbb-a21d-11ed-9efc-5e7820dba16b/kops get instancegroups --name ha-migration.k8s.local -o yaml I0201 11:07:07.582124 5758 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/fc9a7cbb-a21d-11ed-9efc-5e7820dba16b/kops get instancegroups --name ha-migration.k8s.local -o yaml ... skipping 166 lines ... Deleted cluster: "ha-migration.k8s.local" I0201 11:09:25.082019 5758 gcs.go:88] gsutil -u k8s-jkns-gce-soak-1-4 rm -r gs://k8s-jkns-gce-soak-1-4-state-fc I0201 11:09:25.082089 5758 local.go:42] ⚙️ gsutil -u k8s-jkns-gce-soak-1-4 rm -r gs://k8s-jkns-gce-soak-1-4-state-fc Removing gs://k8s-jkns-gce-soak-1-4-state-fc/... I0201 11:09:27.515389 5758 down.go:62] releasing boskos project I0201 11:09:27.523991 5758 boskos.go:83] Boskos heartbeat func received signal to close Error: exit status 1 + EXIT_VALUE=1 + set +o xtrace