e2e-kops-grid-gce-calico-u2004-k23 #1640096486404395008
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 1 failed / 1 succeeded |
| Started | |
| Elapsed | 20m30s |
| Revision | master |
exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest2 Down
... skipping 163 lines ... .......................Updated [https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial]. ..done. WARNING: No host aliases were added to your SSH configs because you do not have any running instances. Try running this command again after running some instances. I0326 21:02:56.239397 6166 up.go:44] Cleaning up any leaked resources from previous cluster I0326 21:02:56.239647 6166 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/48c174b0-cc19-11ed-abe2-dec40ed6026e/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1456704584/key --ssh-user prow I0326 21:02:56.239684 6166 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/48c174b0-cc19-11ed-abe2-dec40ed6026e/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1456704584/key --ssh-user prow Error: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local" not found W0326 21:02:56.478656 6166 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0326 21:02:56.478734 6166 down.go:48] /home/prow/go/src/k8s.io/kops/_rundir/48c174b0-cc19-11ed-abe2-dec40ed6026e/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local --yes I0326 21:02:56.478752 6166 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/48c174b0-cc19-11ed-abe2-dec40ed6026e/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local --yes Error: error reading cluster configuration: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local" not found I0326 21:02:56.610292 6166 gcs.go:51] gsutil ls -b -p e2e-gce-gci-ci-serial gs://e2e-gce-gci-ci-serial-state-48 I0326 21:02:56.610338 6166 local.go:42] ⚙️ gsutil ls -b -p e2e-gce-gci-ci-serial gs://e2e-gce-gci-ci-serial-state-48 I0326 21:02:58.701005 6166 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2023/03/26 21:02:58 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I0326 21:02:58.711507 6166 http.go:37] curl https://ip.jsb.workers.dev I0326 21:02:58.817758 6166 up.go:167] /home/prow/go/src/k8s.io/kops/_rundir/48c174b0-cc19-11ed-abe2-dec40ed6026e/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.23.17 --ssh-public-key /tmp/kops-ssh1456704584/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --gce-service-account=default --admin-access 104.197.84.180/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-west2-c --master-size e2-standard-2 --project e2e-gce-gci-ci-serial I0326 21:02:58.817830 6166 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/48c174b0-cc19-11ed-abe2-dec40ed6026e/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.23.17 --ssh-public-key /tmp/kops-ssh1456704584/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --gce-service-account=default --admin-access 104.197.84.180/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-west2-c --master-size e2-standard-2 --project e2e-gce-gci-ci-serial I0326 21:02:58.872267 6356 create_cluster.go:882] Using SSH public key: /tmp/kops-ssh1456704584/key.pub I0326 21:02:59.009604 6356 new_cluster.go:574] VMs will be configured to use specified Service Account: default I0326 21:02:59.009813 6356 new_cluster.go:1347] Cloud Provider ID: "gce" ... skipping 392 lines ... W0326 21:03:06.088499 6375 vfs_keystorereader.go:143] CA private key was not found I0326 21:03:06.114937 6375 keypair.go:226] Issuing new certificate: "kubernetes-ca" I0326 21:03:06.135933 6375 keypair.go:226] Issuing new certificate: "service-account" I0326 21:03:06.190271 6375 address.go:139] GCE creating address: "api-e2e-e2e-kops-grid-gce-calico-u2004-k23-k8s-local" I0326 21:03:15.934582 6375 executor.go:111] Tasks: 41 done / 67 total; 18 can run I0326 21:03:31.959296 6375 executor.go:111] Tasks: 59 done / 67 total; 6 can run W0326 21:03:36.899186 6375 executor.go:139] error running task "ForwardingRule/api-e2e-e2e-kops-grid-gce-calico-u2004-k23-k8s-local" (9m55s remaining to succeed): error creating ForwardingRule "api-e2e-e2e-kops-grid-gce-calico-u2004-k23-k8s-local": googleapi: Error 400: The resource 'projects/e2e-gce-gci-ci-serial/regions/us-west2/targetPools/api-e2e-e2e-kops-grid-gce-calico-u2004-k23-k8s-local' is not ready, resourceNotReady I0326 21:03:36.899280 6375 executor.go:111] Tasks: 64 done / 67 total; 3 can run I0326 21:04:00.164836 6375 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0326 21:04:00.242236 6375 update_cluster.go:323] Exporting kubeconfig for cluster kOps has set your kubectl context to e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local Cluster is starting. It should be ready in a few minutes. ... skipping 6 lines ... * read about installing addons at: https://kops.sigs.k8s.io/addons. I0326 21:04:10.544905 6166 up.go:251] /home/prow/go/src/k8s.io/kops/_rundir/48c174b0-cc19-11ed-abe2-dec40ed6026e/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local --count 10 --wait 15m0s I0326 21:04:10.544970 6166 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/48c174b0-cc19-11ed-abe2-dec40ed6026e/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local --count 10 --wait 15m0s Validating cluster e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local W0326 21:04:26.242566 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:04:36.296319 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:04:46.348640 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:04:56.396200 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:05:06.443739 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:05:16.492041 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:05:26.546434 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:05:36.595337 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:05:46.643931 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:05:56.697802 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:06:06.751591 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:06:16.802010 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:06:26.851054 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:06:46.908775 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": net/http: TLS handshake timeout W0326 21:07:06.957723 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": net/http: TLS handshake timeout W0326 21:07:17.012269 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:07:27.060154 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:07:47.121377 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": net/http: TLS handshake timeout I0326 21:07:48.072785 6166 boskos.go:86] Sending heartbeat to Boskos W0326 21:07:57.175298 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:08:07.229343 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused W0326 21:08:17.278765 6396 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.236.57.154/api/v1/nodes": dial tcp 35.236.57.154:443: connect: connection refused I0326 21:08:35.056458 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 5 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/control-plane-us-west2-c-rq2l machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/control-plane-us-west2-c-rq2l" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-7psz machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-7psz" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Validation Failed W0326 21:08:35.726007 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:08:46.178852 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Node control-plane-us-west2-c-rq2l node "control-plane-us-west2-c-rq2l" of role "control-plane" is not ready Pod kube-system/kube-controller-manager-control-plane-us-west2-c-rq2l system-cluster-critical pod "kube-controller-manager-control-plane-us-west2-c-rq2l" is not ready (kube-controller-manager) Validation Failed W0326 21:08:46.764374 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:08:57.180785 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Node control-plane-us-west2-c-rq2l node "control-plane-us-west2-c-rq2l" of role "control-plane" is not ready Pod kube-system/kube-controller-manager-control-plane-us-west2-c-rq2l system-cluster-critical pod "kube-controller-manager-control-plane-us-west2-c-rq2l" is not ready (kube-controller-manager) Validation Failed W0326 21:08:58.274043 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:09:08.659469 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 9 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Node control-plane-us-west2-c-rq2l node "control-plane-us-west2-c-rq2l" of role "control-plane" is not ready Pod kube-system/kube-controller-manager-control-plane-us-west2-c-rq2l system-cluster-critical pod "kube-controller-manager-control-plane-us-west2-c-rq2l" is not ready (kube-controller-manager) Pod kube-system/kube-proxy-control-plane-us-west2-c-rq2l system-node-critical pod "kube-proxy-control-plane-us-west2-c-rq2l" is pending Validation Failed W0326 21:09:09.274276 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:09:19.629323 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 9 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Node control-plane-us-west2-c-rq2l node "control-plane-us-west2-c-rq2l" of role "control-plane" is not ready Pod kube-system/kube-controller-manager-control-plane-us-west2-c-rq2l system-cluster-critical pod "kube-controller-manager-control-plane-us-west2-c-rq2l" is not ready (kube-controller-manager) Pod kube-system/kube-scheduler-control-plane-us-west2-c-rq2l system-cluster-critical pod "kube-scheduler-control-plane-us-west2-c-rq2l" is pending Validation Failed W0326 21:09:20.358866 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:09:30.756929 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 10 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Node control-plane-us-west2-c-rq2l node "control-plane-us-west2-c-rq2l" of role "control-plane" is not ready Pod kube-system/etcd-manager-events-control-plane-us-west2-c-rq2l system-cluster-critical pod "etcd-manager-events-control-plane-us-west2-c-rq2l" is pending Pod kube-system/kube-apiserver-control-plane-us-west2-c-rq2l system-cluster-critical pod "kube-apiserver-control-plane-us-west2-c-rq2l" is pending Pod kube-system/kube-controller-manager-control-plane-us-west2-c-rq2l system-cluster-critical pod "kube-controller-manager-control-plane-us-west2-c-rq2l" is not ready (kube-controller-manager) Validation Failed W0326 21:09:31.333856 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:09:41.691401 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Node control-plane-us-west2-c-rq2l node "control-plane-us-west2-c-rq2l" of role "control-plane" is not ready Pod kube-system/kube-controller-manager-control-plane-us-west2-c-rq2l system-cluster-critical pod "kube-controller-manager-control-plane-us-west2-c-rq2l" is not ready (kube-controller-manager) Validation Failed W0326 21:09:42.264183 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:09:52.630206 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Node control-plane-us-west2-c-rq2l node "control-plane-us-west2-c-rq2l" of role "control-plane" is not ready Pod kube-system/kube-controller-manager-control-plane-us-west2-c-rq2l system-cluster-critical pod "kube-controller-manager-control-plane-us-west2-c-rq2l" is not ready (kube-controller-manager) Validation Failed W0326 21:09:53.488211 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:10:03.823521 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-7psz machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-7psz" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Node control-plane-us-west2-c-rq2l node "control-plane-us-west2-c-rq2l" of role "control-plane" is not ready Validation Failed W0326 21:10:04.571938 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:10:14.903172 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-7psz machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-7psz" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Node control-plane-us-west2-c-rq2l node "control-plane-us-west2-c-rq2l" of role "control-plane" is not ready Validation Failed W0326 21:10:15.721847 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:10:26.122117 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 12 lines ... Pod kube-system/calico-kube-controllers-795c657547-942lf system-cluster-critical pod "calico-kube-controllers-795c657547-942lf" is pending Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is pending Pod kube-system/coredns-5d479c4f64-gv44d system-cluster-critical pod "coredns-5d479c4f64-gv44d" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Pod kube-system/dns-controller-6f69f47785-b6w4d system-cluster-critical pod "dns-controller-6f69f47785-b6w4d" is pending Validation Failed W0326 21:10:27.227468 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:10:37.563766 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 12 lines ... Pod kube-system/calico-kube-controllers-795c657547-942lf system-cluster-critical pod "calico-kube-controllers-795c657547-942lf" is pending Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is pending Pod kube-system/coredns-5d479c4f64-gv44d system-cluster-critical pod "coredns-5d479c4f64-gv44d" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Pod kube-system/dns-controller-6f69f47785-b6w4d system-cluster-critical pod "dns-controller-6f69f47785-b6w4d" is pending Validation Failed W0326 21:10:38.569494 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:10:49.009221 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 11 lines ... Pod kube-system/calico-kube-controllers-795c657547-942lf system-cluster-critical pod "calico-kube-controllers-795c657547-942lf" is pending Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Pod kube-system/coredns-5d479c4f64-gv44d system-cluster-critical pod "coredns-5d479c4f64-gv44d" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Pod kube-system/dns-controller-6f69f47785-b6w4d system-cluster-critical pod "dns-controller-6f69f47785-b6w4d" is pending Validation Failed W0326 21:10:49.895907 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:11:00.331345 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Pod kube-system/coredns-5d479c4f64-gv44d system-cluster-critical pod "coredns-5d479c4f64-gv44d" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Validation Failed W0326 21:11:01.189007 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:11:11.588177 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Pod kube-system/coredns-5d479c4f64-gv44d system-cluster-critical pod "coredns-5d479c4f64-gv44d" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Validation Failed W0326 21:11:12.344876 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:11:22.753403 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Pod kube-system/coredns-5d479c4f64-gv44d system-cluster-critical pod "coredns-5d479c4f64-gv44d" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Validation Failed W0326 21:11:23.602509 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:11:34.077643 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Pod kube-system/coredns-5d479c4f64-gv44d system-cluster-critical pod "coredns-5d479c4f64-gv44d" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Validation Failed W0326 21:11:34.861788 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:11:45.240654 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Pod kube-system/coredns-5d479c4f64-gv44d system-cluster-critical pod "coredns-5d479c4f64-gv44d" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Validation Failed W0326 21:11:45.953632 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:11:56.392563 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Pod kube-system/coredns-5d479c4f64-gv44d system-cluster-critical pod "coredns-5d479c4f64-gv44d" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Validation Failed W0326 21:11:57.228295 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:12:07.572310 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Pod kube-system/coredns-5d479c4f64-gv44d system-cluster-critical pod "coredns-5d479c4f64-gv44d" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Validation Failed W0326 21:12:08.175307 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:12:18.518873 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Pod kube-system/coredns-5d479c4f64-gv44d system-cluster-critical pod "coredns-5d479c4f64-gv44d" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Validation Failed W0326 21:12:19.363156 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:12:29.756586 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-g43g" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-j8vh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial/zones/us-west2-c/instances/nodes-us-west2-c-qt0m" has not yet joined cluster Pod kube-system/coredns-5d479c4f64-gv44d system-cluster-critical pod "coredns-5d479c4f64-gv44d" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Validation Failed W0326 21:12:30.543214 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:12:41.000354 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 20 lines ... Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Pod kube-system/metadata-proxy-v0.12-dk4gw system-node-critical pod "metadata-proxy-v0.12-dk4gw" is pending Pod kube-system/metadata-proxy-v0.12-h9njf system-node-critical pod "metadata-proxy-v0.12-h9njf" is pending Pod kube-system/metadata-proxy-v0.12-k6gt6 system-node-critical pod "metadata-proxy-v0.12-k6gt6" is pending Pod kube-system/metadata-proxy-v0.12-nxgbr system-node-critical pod "metadata-proxy-v0.12-nxgbr" is pending Validation Failed W0326 21:12:42.050387 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:12:48.085940 6166 boskos.go:86] Sending heartbeat to Boskos I0326 21:12:52.413504 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 ... skipping 21 lines ... Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Pod kube-system/metadata-proxy-v0.12-dk4gw system-node-critical pod "metadata-proxy-v0.12-dk4gw" is pending Pod kube-system/metadata-proxy-v0.12-h9njf system-node-critical pod "metadata-proxy-v0.12-h9njf" is pending Pod kube-system/metadata-proxy-v0.12-k6gt6 system-node-critical pod "metadata-proxy-v0.12-k6gt6" is pending Pod kube-system/metadata-proxy-v0.12-nxgbr system-node-critical pod "metadata-proxy-v0.12-nxgbr" is pending Validation Failed W0326 21:12:52.988566 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:13:03.400324 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 20 lines ... Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Pod kube-system/metadata-proxy-v0.12-dk4gw system-node-critical pod "metadata-proxy-v0.12-dk4gw" is pending Pod kube-system/metadata-proxy-v0.12-h9njf system-node-critical pod "metadata-proxy-v0.12-h9njf" is pending Pod kube-system/metadata-proxy-v0.12-k6gt6 system-node-critical pod "metadata-proxy-v0.12-k6gt6" is pending Pod kube-system/metadata-proxy-v0.12-nxgbr system-node-critical pod "metadata-proxy-v0.12-nxgbr" is pending Validation Failed W0326 21:13:04.012555 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:13:14.356125 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 18 lines ... Pod kube-system/coredns-5d479c4f64-gv44d system-cluster-critical pod "coredns-5d479c4f64-gv44d" is pending Pod kube-system/coredns-autoscaler-557ccb4c66-gwwq9 system-cluster-critical pod "coredns-autoscaler-557ccb4c66-gwwq9" is pending Pod kube-system/metadata-proxy-v0.12-dk4gw system-node-critical pod "metadata-proxy-v0.12-dk4gw" is pending Pod kube-system/metadata-proxy-v0.12-h9njf system-node-critical pod "metadata-proxy-v0.12-h9njf" is pending Pod kube-system/metadata-proxy-v0.12-k6gt6 system-node-critical pod "metadata-proxy-v0.12-k6gt6" is pending Validation Failed W0326 21:13:14.955121 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:13:25.294361 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 15 lines ... Pod kube-system/calico-node-pn56z system-node-critical pod "calico-node-pn56z" is not ready (calico-node) Pod kube-system/calico-node-vj27k system-node-critical pod "calico-node-vj27k" is pending Pod kube-system/calico-node-zc2ck system-node-critical pod "calico-node-zc2ck" is pending Pod kube-system/metadata-proxy-v0.12-dk4gw system-node-critical pod "metadata-proxy-v0.12-dk4gw" is pending Pod kube-system/metadata-proxy-v0.12-h9njf system-node-critical pod "metadata-proxy-v0.12-h9njf" is pending Validation Failed W0326 21:13:25.864594 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:13:36.288904 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 13 lines ... Pod kube-system/calico-node-fzxl7 system-node-critical pod "calico-node-fzxl7" is pending Pod kube-system/calico-node-pn56z system-node-critical pod "calico-node-pn56z" is not ready (calico-node) Pod kube-system/calico-node-vj27k system-node-critical pod "calico-node-vj27k" is pending Pod kube-system/calico-node-zc2ck system-node-critical pod "calico-node-zc2ck" is not ready (calico-node) Pod kube-system/metadata-proxy-v0.12-h9njf system-node-critical pod "metadata-proxy-v0.12-h9njf" is pending Validation Failed W0326 21:13:36.936352 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:13:47.351754 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 10 lines ... KIND NAME MESSAGE Pod kube-system/calico-node-fzxl7 system-node-critical pod "calico-node-fzxl7" is not ready (calico-node) Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Pod kube-system/calico-node-vj27k system-node-critical pod "calico-node-vj27k" is not ready (calico-node) Pod kube-system/calico-node-zc2ck system-node-critical pod "calico-node-zc2ck" is not ready (calico-node) Validation Failed W0326 21:13:47.964945 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:13:58.300272 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 10 lines ... KIND NAME MESSAGE Pod kube-system/calico-node-fzxl7 system-node-critical pod "calico-node-fzxl7" is not ready (calico-node) Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Pod kube-system/calico-node-vj27k system-node-critical pod "calico-node-vj27k" is not ready (calico-node) Pod kube-system/calico-node-zc2ck system-node-critical pod "calico-node-zc2ck" is not ready (calico-node) Validation Failed W0326 21:13:58.928136 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:14:09.282201 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 9 lines ... VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fzxl7 system-node-critical pod "calico-node-fzxl7" is not ready (calico-node) Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Pod kube-system/calico-node-vj27k system-node-critical pod "calico-node-vj27k" is not ready (calico-node) Validation Failed W0326 21:14:10.154770 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:14:20.544946 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 8 lines ... VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Pod kube-system/kube-proxy-nodes-us-west2-c-j8vh system-node-critical pod "kube-proxy-nodes-us-west2-c-j8vh" is pending Validation Failed W0326 21:14:21.344173 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:14:31.683687 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:14:32.254261 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:14:42.646699 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:14:43.272895 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:14:53.625093 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:14:54.213739 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:15:04.553817 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:15:05.443105 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:15:15.789976 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:15:16.611468 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:15:26.926177 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:15:27.726442 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:15:38.080604 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:15:38.821029 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:15:49.326183 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:15:50.076023 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:16:00.484732 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:16:01.274279 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:16:11.732881 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:16:12.575251 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:16:23.028198 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:16:23.596517 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:16:33.987431 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:16:34.603361 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:16:45.114484 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:16:45.846445 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:16:56.210770 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:16:57.226853 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:17:07.585707 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:17:08.172966 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:17:18.502068 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:17:19.099552 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:17:29.552076 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:17:30.655769 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:17:41.069773 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:17:42.130076 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:17:48.098109 6166 boskos.go:86] Sending heartbeat to Boskos I0326 21:17:52.496765 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 ... skipping 8 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:17:53.096248 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:18:03.555909 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:18:04.471451 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:18:14.832226 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:18:15.400967 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:18:25.763675 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:18:26.811026 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:18:37.222954 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:18:37.906309 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:18:48.343966 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:18:49.058558 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:18:59.414122 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:18:59.963481 6396 validate_cluster.go:232] (will retry): cluster not yet healthy I0326 21:19:10.321497 6396 gce_cloud.go:301] Scanning zones: [us-west2-a us-west2-b us-west2-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west2-c ControlPlane e2-standard-2 1 1 us-west2 nodes-us-west2-c Node e2-medium 4 4 us-west2 ... skipping 7 lines ... nodes-us-west2-c-qt0m node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tlhzk system-node-critical pod "calico-node-tlhzk" is not ready (calico-node) Validation Failed W0326 21:19:11.020364 6396 validate_cluster.go:232] (will retry): cluster not yet healthy Error: validation failed: wait time exceeded during validation I0326 21:19:21.026221 6166 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/48c174b0-cc19-11ed-abe2-dec40ed6026e/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1456704584/key --ssh-user prow I0326 21:19:21.026279 6166 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/48c174b0-cc19-11ed-abe2-dec40ed6026e/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1456704584/key --ssh-user prow I0326 21:19:21.434188 6407 gce.go:99] Scanning zones: [us-west2-a us-west2-b us-west2-c] 2023/03/26 21:19:26 Dumping node control-plane-us-west2-c-rq2l 2023/03/26 21:19:32 error dumping node control-plane-us-west2-c-rq2l: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/03/26 21:19:32 error dumping node control-plane-us-west2-c-rq2l: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/03/26 21:19:32 Dumping node nodes-us-west2-c-7psz 2023/03/26 21:19:37 Dumping node nodes-us-west2-c-g43g 2023/03/26 21:19:41 Dumping node nodes-us-west2-c-j8vh 2023/03/26 21:19:45 Dumping node nodes-us-west2-c-qt0m I0326 21:19:49.522072 6166 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/48c174b0-cc19-11ed-abe2-dec40ed6026e/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local -o yaml I0326 21:19:49.522125 6166 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/48c174b0-cc19-11ed-abe2-dec40ed6026e/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local -o yaml ... skipping 144 lines ... Deleted cluster: "e2e-e2e-kops-grid-gce-calico-u2004-k23.k8s.local" I0326 21:22:01.805498 6166 gcs.go:88] gsutil -u e2e-gce-gci-ci-serial rm -r gs://e2e-gce-gci-ci-serial-state-48 I0326 21:22:01.805569 6166 local.go:42] ⚙️ gsutil -u e2e-gce-gci-ci-serial rm -r gs://e2e-gce-gci-ci-serial-state-48 Removing gs://e2e-gce-gci-ci-serial-state-48/... I0326 21:22:05.176183 6166 down.go:62] releasing boskos project I0326 21:22:05.189067 6166 boskos.go:83] Boskos heartbeat func received signal to close Error: exit status 1 + EXIT_VALUE=1 + set +o xtrace