e2e-kops-grid-gce-calico-u2004-k26 #1616985927572262912
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 1 failed / 1 succeeded |
| Started | |
| Elapsed | 20m5s |
| Revision | master |
exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest2 Down
... skipping 163 lines ... ............................Updated [https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu]. .done. WARNING: No host aliases were added to your SSH configs because you do not have any running instances. Try running this command again after running some instances. I0122 02:29:39.977969 5506 up.go:44] Cleaning up any leaked resources from previous cluster I0122 02:29:39.978108 5506 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/58d0d775-99fc-11ed-a697-56ea552f9d82/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1136565177/key --ssh-user prow I0122 02:29:39.978138 5506 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/58d0d775-99fc-11ed-a697-56ea552f9d82/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1136565177/key --ssh-user prow Error: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" not found W0122 02:29:40.291682 5506 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0122 02:29:40.291736 5506 down.go:48] /home/prow/go/src/k8s.io/kops/_rundir/58d0d775-99fc-11ed-a697-56ea552f9d82/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --yes I0122 02:29:40.291751 5506 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/58d0d775-99fc-11ed-a697-56ea552f9d82/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --yes Error: error reading cluster configuration: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" not found I0122 02:29:40.503520 5506 gcs.go:51] gsutil ls -b -p gce-up-c1-3-g1-4-up-clu gs://gce-up-c1-3-g1-4-up-clu-state-58 I0122 02:29:40.503563 5506 local.go:42] ⚙️ gsutil ls -b -p gce-up-c1-3-g1-4-up-clu gs://gce-up-c1-3-g1-4-up-clu-state-58 I0122 02:29:42.580764 5506 gcs.go:70] gsutil mb -p gce-up-c1-3-g1-4-up-clu gs://gce-up-c1-3-g1-4-up-clu-state-58 I0122 02:29:42.580812 5506 local.go:42] ⚙️ gsutil mb -p gce-up-c1-3-g1-4-up-clu gs://gce-up-c1-3-g1-4-up-clu-state-58 Creating gs://gce-up-c1-3-g1-4-up-clu-state-58/... I0122 02:29:45.191133 5506 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2023/01/22 02:29:45 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I0122 02:29:45.203766 5506 http.go:37] curl https://ip.jsb.workers.dev I0122 02:29:45.313892 5506 up.go:167] /home/prow/go/src/k8s.io/kops/_rundir/58d0d775-99fc-11ed-a697-56ea552f9d82/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.26.1 --ssh-public-key /tmp/kops-ssh1136565177/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 34.132.41.60/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-central1-a --master-size e2-standard-2 --project gce-up-c1-3-g1-4-up-clu I0122 02:29:45.313935 5506 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/58d0d775-99fc-11ed-a697-56ea552f9d82/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.26.1 --ssh-public-key /tmp/kops-ssh1136565177/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 34.132.41.60/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-central1-a --master-size e2-standard-2 --project gce-up-c1-3-g1-4-up-clu I0122 02:29:45.370262 5829 create_cluster.go:882] Using SSH public key: /tmp/kops-ssh1136565177/key.pub I0122 02:29:45.733592 5829 new_cluster.go:573] VMs will be configured to use specified Service Account: default I0122 02:29:45.733705 5829 new_cluster.go:1338] Cloud Provider ID: "gce" ... skipping 349 lines ... I0122 02:29:53.876554 5847 keypair.go:226] Issuing new certificate: "etcd-manager-ca-events" I0122 02:29:53.876601 5847 keypair.go:226] Issuing new certificate: "apiserver-aggregator-ca" W0122 02:29:53.969745 5847 vfs_keystorereader.go:143] CA private key was not found I0122 02:29:54.076360 5847 keypair.go:226] Issuing new certificate: "service-account" I0122 02:29:54.076362 5847 keypair.go:226] Issuing new certificate: "kubernetes-ca" I0122 02:30:06.653414 5847 executor.go:111] Tasks: 42 done / 67 total; 19 can run W0122 02:30:19.704257 5847 executor.go:139] error running task "ForwardingRule/api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local" (9m46s remaining to succeed): error creating ForwardingRule "api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local": googleapi: Error 400: The resource 'projects/gce-up-c1-3-g1-4-up-clu/regions/us-central1/targetPools/api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local' is not ready, resourceNotReady I0122 02:30:19.704349 5847 executor.go:111] Tasks: 60 done / 67 total; 5 can run I0122 02:30:26.478913 5847 executor.go:111] Tasks: 65 done / 67 total; 2 can run I0122 02:30:40.479305 5847 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0122 02:30:40.548574 5847 update_cluster.go:323] Exporting kubeconfig for cluster kOps has set your kubectl context to e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local ... skipping 7 lines ... * read about installing addons at: https://kops.sigs.k8s.io/addons. I0122 02:30:50.929825 5506 up.go:251] /home/prow/go/src/k8s.io/kops/_rundir/58d0d775-99fc-11ed-a697-56ea552f9d82/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --count 10 --wait 15m0s I0122 02:30:50.929901 5506 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/58d0d775-99fc-11ed-a697-56ea552f9d82/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --count 10 --wait 15m0s Validating cluster e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local W0122 02:31:21.245437 5867 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.224.104.215/api/v1/nodes": dial tcp 35.224.104.215:443: i/o timeout W0122 02:32:01.247505 5867 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.224.104.215/api/v1/nodes": dial tcp 35.224.104.215:443: i/o timeout W0122 02:32:11.252266 5867 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.224.104.215/api/v1/nodes": dial tcp 35.224.104.215:443: connect: connection refused W0122 02:32:21.256006 5867 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.224.104.215/api/v1/nodes": dial tcp 35.224.104.215:443: connect: connection refused W0122 02:32:31.259630 5867 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.224.104.215/api/v1/nodes": dial tcp 35.224.104.215:443: connect: connection refused W0122 02:32:41.262407 5867 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.224.104.215/api/v1/nodes": dial tcp 35.224.104.215:443: connect: connection refused W0122 02:32:51.265918 5867 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.224.104.215/api/v1/nodes": dial tcp 35.224.104.215:443: connect: connection refused W0122 02:33:01.269153 5867 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.224.104.215/api/v1/nodes": dial tcp 35.224.104.215:443: connect: connection refused W0122 02:33:11.271973 5867 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.224.104.215/api/v1/nodes": dial tcp 35.224.104.215:443: connect: connection refused W0122 02:33:21.275224 5867 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.224.104.215/api/v1/nodes": dial tcp 35.224.104.215:443: connect: connection refused W0122 02:33:41.278206 5867 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.224.104.215/api/v1/nodes": net/http: TLS handshake timeout W0122 02:33:51.280310 5867 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.224.104.215/api/v1/nodes": dial tcp 35.224.104.215:443: connect: connection refused W0122 02:34:11.284049 5867 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://35.224.104.215/api/v1/nodes": net/http: TLS handshake timeout I0122 02:34:21.738797 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 5 lines ... Machine https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/control-plane-us-central1-a-s6kq machine "https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/control-plane-us-central1-a-s6kq" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-8ljq machine "https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-8ljq" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-mvpf machine "https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-mvpf" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-s6xt machine "https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-s6xt" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-tzcc machine "https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-tzcc" has not yet joined cluster Validation Failed W0122 02:34:22.359804 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:34:30.827583 5506 boskos.go:86] Sending heartbeat to Boskos I0122 02:34:32.774371 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-8ljq machine "https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-8ljq" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-mvpf machine "https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-mvpf" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-s6xt machine "https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-s6xt" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-tzcc machine "https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-tzcc" has not yet joined cluster Pod kube-system/kube-controller-manager-control-plane-us-central1-a-s6kq system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-a-s6kq" is not ready (kube-controller-manager) Validation Failed W0122 02:34:33.376394 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:34:43.828447 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... Machine https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-8ljq machine "https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-8ljq" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-mvpf machine "https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-mvpf" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-s6xt machine "https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-s6xt" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-tzcc machine "https://www.googleapis.com/compute/v1/projects/gce-up-c1-3-g1-4-up-clu/zones/us-central1-a/instances/nodes-us-central1-a-tzcc" has not yet joined cluster Pod kube-system/kube-scheduler-control-plane-us-central1-a-s6kq system-cluster-critical pod "kube-scheduler-control-plane-us-central1-a-s6kq" is pending Validation Failed W0122 02:34:44.338999 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:34:54.760797 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 13 lines ... Pod kube-system/cloud-controller-manager-2vw4c system-cluster-critical pod "cloud-controller-manager-2vw4c" is pending Pod kube-system/coredns-7bc44848df-8g57x system-cluster-critical pod "coredns-7bc44848df-8g57x" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-bd9s8 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-bd9s8" is pending Pod kube-system/dns-controller-6d5f4bf5d6-bnttl system-cluster-critical pod "dns-controller-6d5f4bf5d6-bnttl" is pending Pod kube-system/kops-controller-5xrcc system-cluster-critical pod "kops-controller-5xrcc" is pending Validation Failed W0122 02:34:55.365435 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:35:05.772717 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 11 lines ... Pod kube-system/calico-kube-controllers-55f488656-nsfl9 system-cluster-critical pod "calico-kube-controllers-55f488656-nsfl9" is pending Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is pending Pod kube-system/cloud-controller-manager-2vw4c system-cluster-critical pod "cloud-controller-manager-2vw4c" is pending Pod kube-system/coredns-7bc44848df-8g57x system-cluster-critical pod "coredns-7bc44848df-8g57x" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-bd9s8 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-bd9s8" is pending Validation Failed W0122 02:35:06.356970 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:35:16.776826 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 16 lines ... Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is pending Pod kube-system/coredns-7bc44848df-8g57x system-cluster-critical pod "coredns-7bc44848df-8g57x" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-bd9s8 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-bd9s8" is pending Pod kube-system/etcd-manager-main-control-plane-us-central1-a-s6kq system-cluster-critical pod "etcd-manager-main-control-plane-us-central1-a-s6kq" is pending Pod kube-system/kube-proxy-nodes-us-central1-a-s6xt system-node-critical pod "kube-proxy-nodes-us-central1-a-s6xt" is pending Validation Failed W0122 02:35:17.306069 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:35:27.785698 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 17 lines ... Pod kube-system/calico-node-2ldrj system-node-critical pod "calico-node-2ldrj" is pending Pod kube-system/calico-node-5jn46 system-node-critical pod "calico-node-5jn46" is pending Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is pending Pod kube-system/coredns-7bc44848df-8g57x system-cluster-critical pod "coredns-7bc44848df-8g57x" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-bd9s8 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-bd9s8" is pending Validation Failed W0122 02:35:28.328471 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:35:38.826340 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 12 lines ... Pod kube-system/calico-node-2c9fq system-node-critical pod "calico-node-2c9fq" is not ready (calico-node) Pod kube-system/calico-node-2ldrj system-node-critical pod "calico-node-2ldrj" is pending Pod kube-system/calico-node-5jn46 system-node-critical pod "calico-node-5jn46" is pending Pod kube-system/coredns-7bc44848df-8g57x system-cluster-critical pod "coredns-7bc44848df-8g57x" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-bd9s8 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-bd9s8" is pending Validation Failed W0122 02:35:39.356708 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:35:49.730441 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 11 lines ... Pod kube-system/calico-node-27wh9 system-node-critical pod "calico-node-27wh9" is not ready (calico-node) Pod kube-system/calico-node-2c9fq system-node-critical pod "calico-node-2c9fq" is not ready (calico-node) Pod kube-system/calico-node-2ldrj system-node-critical pod "calico-node-2ldrj" is not ready (calico-node) Pod kube-system/calico-node-5jn46 system-node-critical pod "calico-node-5jn46" is not ready (calico-node) Pod kube-system/coredns-7bc44848df-bbmp4 system-cluster-critical pod "coredns-7bc44848df-bbmp4" is pending Validation Failed W0122 02:35:50.241376 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:36:00.628748 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 10 lines ... KIND NAME MESSAGE Pod kube-system/calico-node-27wh9 system-node-critical pod "calico-node-27wh9" is not ready (calico-node) Pod kube-system/calico-node-2c9fq system-node-critical pod "calico-node-2c9fq" is not ready (calico-node) Pod kube-system/calico-node-2ldrj system-node-critical pod "calico-node-2ldrj" is not ready (calico-node) Pod kube-system/calico-node-5jn46 system-node-critical pod "calico-node-5jn46" is not ready (calico-node) Validation Failed W0122 02:36:01.173339 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:36:11.610783 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 10 lines ... KIND NAME MESSAGE Pod kube-system/calico-node-27wh9 system-node-critical pod "calico-node-27wh9" is not ready (calico-node) Pod kube-system/calico-node-2ldrj system-node-critical pod "calico-node-2ldrj" is not ready (calico-node) Pod kube-system/calico-node-5jn46 system-node-critical pod "calico-node-5jn46" is not ready (calico-node) Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:36:12.142107 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:36:22.562910 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:36:23.098719 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:36:33.528381 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:36:34.066604 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:36:44.542346 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:36:45.198452 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:36:55.619295 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:36:56.111028 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:37:06.545312 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:37:07.156114 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:37:17.580434 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:37:18.107178 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:37:28.510327 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:37:29.030965 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:37:39.468246 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:37:39.974637 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:37:50.353764 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:37:50.872211 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:38:01.264508 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:38:01.811304 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:38:12.262124 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:38:12.776166 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:38:23.256005 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:38:23.808094 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:38:34.183535 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:38:34.648168 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:38:45.014120 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:38:45.545008 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:38:55.959532 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:38:56.480722 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:39:06.891704 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:39:07.423655 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:39:17.777365 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:39:18.353311 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:39:28.730018 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:39:29.247549 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:39:30.839497 5506 boskos.go:86] Sending heartbeat to Boskos I0122 02:39:39.601629 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 8 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:39:40.131213 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:39:50.487006 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:39:50.963580 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:40:01.337269 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:40:01.848715 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:40:12.262034 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:40:12.793408 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:40:23.195064 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:40:23.707211 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:40:34.119885 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:40:34.720461 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:40:45.097919 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:40:45.660184 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:40:56.109636 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:40:56.640369 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:41:07.031563 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:41:07.571106 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:41:17.954067 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:41:18.466509 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:41:28.851908 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:41:29.342964 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:41:39.810980 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:41:40.313738 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:41:50.719646 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:41:51.212781 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:42:01.575845 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:42:02.154025 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:42:12.607750 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:42:13.124934 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:42:23.647493 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:42:24.227585 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:42:34.615562 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:42:35.178678 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:42:45.646432 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:42:46.163810 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:42:56.571588 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:42:57.178798 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:43:07.599959 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:43:08.122942 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:43:18.471956 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:43:18.992730 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:43:29.398953 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:43:29.901332 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:43:40.312677 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:43:40.906447 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:43:51.253089 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:43:51.809873 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:44:02.226262 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:44:02.803528 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:44:13.178894 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:44:13.760884 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:44:24.131093 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:44:24.708403 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:44:30.850503 5506 boskos.go:86] Sending heartbeat to Boskos I0122 02:44:35.151409 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 8 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:44:35.615530 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:44:46.089518 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:44:46.580430 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:44:56.985478 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:44:57.484085 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:45:07.855768 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:45:08.454504 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:45:18.873040 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:45:19.393665 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:45:29.800522 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:45:30.342220 5867 validate_cluster.go:232] (will retry): cluster not yet healthy I0122 02:45:40.734546 5867 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-tzcc node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-prw2c system-node-critical pod "calico-node-prw2c" is not ready (calico-node) Validation Failed W0122 02:45:41.299250 5867 validate_cluster.go:232] (will retry): cluster not yet healthy Error: validation failed: wait time exceeded during validation I0122 02:45:51.305888 5506 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/58d0d775-99fc-11ed-a697-56ea552f9d82/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1136565177/key --ssh-user prow I0122 02:45:51.305952 5506 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/58d0d775-99fc-11ed-a697-56ea552f9d82/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1136565177/key --ssh-user prow I0122 02:45:51.766401 5878 gce.go:99] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] 2023/01/22 02:45:55 Dumping node control-plane-us-central1-a-s6kq 2023/01/22 02:45:58 error dumping node control-plane-us-central1-a-s6kq: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/01/22 02:45:58 error dumping node control-plane-us-central1-a-s6kq: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/01/22 02:45:58 Dumping node nodes-us-central1-a-8ljq 2023/01/22 02:46:01 Dumping node nodes-us-central1-a-mvpf 2023/01/22 02:46:03 Dumping node nodes-us-central1-a-s6xt 2023/01/22 02:46:05 Dumping node nodes-us-central1-a-tzcc I0122 02:46:07.577742 5506 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/58d0d775-99fc-11ed-a697-56ea552f9d82/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local -o yaml I0122 02:46:07.577850 5506 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/58d0d775-99fc-11ed-a697-56ea552f9d82/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local -o yaml ... skipping 156 lines ... Deleted cluster: "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" I0122 02:48:28.589080 5506 gcs.go:88] gsutil -u gce-up-c1-3-g1-4-up-clu rm -r gs://gce-up-c1-3-g1-4-up-clu-state-58 I0122 02:48:28.589155 5506 local.go:42] ⚙️ gsutil -u gce-up-c1-3-g1-4-up-clu rm -r gs://gce-up-c1-3-g1-4-up-clu-state-58 Removing gs://gce-up-c1-3-g1-4-up-clu-state-58/... I0122 02:48:31.289777 5506 down.go:62] releasing boskos project I0122 02:48:31.304521 5506 boskos.go:83] Boskos heartbeat func received signal to close Error: exit status 1 + EXIT_VALUE=1 + set +o xtrace