e2e-kops-grid-gce-calico-u2004-k26 #1615536226536787968
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 1 failed / 1 succeeded |
| Started | |
| Elapsed | 19m56s |
| Revision | master |
exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest2 Down
... skipping 163 lines ... ...............................Updated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log]. ..done. WARNING: No host aliases were added to your SSH configs because you do not have any running instances. Try running this command again after running some instances. I0118 02:29:28.472498 5471 up.go:44] Cleaning up any leaked resources from previous cluster I0118 02:29:28.472609 5471 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/99e826ad-96d7-11ed-824d-f64c9135b4ea/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh299717887/key --ssh-user prow I0118 02:29:28.472629 5471 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/99e826ad-96d7-11ed-824d-f64c9135b4ea/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh299717887/key --ssh-user prow Error: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" not found W0118 02:29:28.706185 5471 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0118 02:29:28.706223 5471 down.go:48] /home/prow/go/src/k8s.io/kops/_rundir/99e826ad-96d7-11ed-824d-f64c9135b4ea/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --yes I0118 02:29:28.706234 5471 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/99e826ad-96d7-11ed-824d-f64c9135b4ea/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --yes Error: error reading cluster configuration: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" not found I0118 02:29:28.870337 5471 gcs.go:51] gsutil ls -b -p k8s-jkns-gci-gce-sd-log gs://k8s-jkns-gci-gce-sd-log-state-99 I0118 02:29:28.870370 5471 local.go:42] ⚙️ gsutil ls -b -p k8s-jkns-gci-gce-sd-log gs://k8s-jkns-gci-gce-sd-log-state-99 I0118 02:29:30.827709 5471 gcs.go:70] gsutil mb -p k8s-jkns-gci-gce-sd-log gs://k8s-jkns-gci-gce-sd-log-state-99 I0118 02:29:30.827751 5471 local.go:42] ⚙️ gsutil mb -p k8s-jkns-gci-gce-sd-log gs://k8s-jkns-gci-gce-sd-log-state-99 Creating gs://k8s-jkns-gci-gce-sd-log-state-99/... I0118 02:29:33.120127 5471 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2023/01/18 02:29:33 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I0118 02:29:33.133342 5471 http.go:37] curl https://ip.jsb.workers.dev I0118 02:29:33.233937 5471 up.go:167] /home/prow/go/src/k8s.io/kops/_rundir/99e826ad-96d7-11ed-824d-f64c9135b4ea/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.26.0 --ssh-public-key /tmp/kops-ssh299717887/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 34.68.127.27/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-central1-a --master-size e2-standard-2 --project k8s-jkns-gci-gce-sd-log I0118 02:29:33.233973 5471 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/99e826ad-96d7-11ed-824d-f64c9135b4ea/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.26.0 --ssh-public-key /tmp/kops-ssh299717887/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 34.68.127.27/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-central1-a --master-size e2-standard-2 --project k8s-jkns-gci-gce-sd-log I0118 02:29:33.285872 5794 create_cluster.go:882] Using SSH public key: /tmp/kops-ssh299717887/key.pub I0118 02:29:33.547545 5794 new_cluster.go:573] VMs will be configured to use specified Service Account: default I0118 02:29:33.547676 5794 new_cluster.go:1338] Cloud Provider ID: "gce" ... skipping 349 lines ... I0118 02:29:42.587812 5815 keypair.go:226] Issuing new certificate: "etcd-peers-ca-events" I0118 02:29:42.593324 5815 keypair.go:226] Issuing new certificate: "etcd-manager-ca-events" W0118 02:29:42.785598 5815 vfs_keystorereader.go:143] CA private key was not found I0118 02:29:42.867885 5815 keypair.go:226] Issuing new certificate: "service-account" I0118 02:29:42.971720 5815 keypair.go:226] Issuing new certificate: "kubernetes-ca" I0118 02:29:58.270582 5815 executor.go:111] Tasks: 42 done / 67 total; 19 can run W0118 02:30:08.307418 5815 executor.go:139] error running task "ForwardingRule/api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local" (9m49s remaining to succeed): error creating ForwardingRule "api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local": googleapi: Error 400: The resource 'projects/k8s-jkns-gci-gce-sd-log/regions/us-central1/targetPools/api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local' is not ready, resourceNotReady I0118 02:30:08.307625 5815 executor.go:111] Tasks: 60 done / 67 total; 5 can run I0118 02:30:15.139445 5815 executor.go:111] Tasks: 65 done / 67 total; 2 can run I0118 02:30:28.145930 5815 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0118 02:30:28.558911 5815 update_cluster.go:323] Exporting kubeconfig for cluster kOps has set your kubectl context to e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local ... skipping 7 lines ... * read about installing addons at: https://kops.sigs.k8s.io/addons. I0118 02:30:38.863231 5471 up.go:251] /home/prow/go/src/k8s.io/kops/_rundir/99e826ad-96d7-11ed-824d-f64c9135b4ea/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --count 10 --wait 15m0s I0118 02:30:38.863277 5471 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/99e826ad-96d7-11ed-824d-f64c9135b4ea/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --count 10 --wait 15m0s Validating cluster e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local W0118 02:31:09.514739 5835 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.170.211.246/api/v1/nodes": dial tcp 34.170.211.246:443: i/o timeout W0118 02:31:49.515506 5835 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.170.211.246/api/v1/nodes": dial tcp 34.170.211.246:443: i/o timeout W0118 02:31:59.517733 5835 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.170.211.246/api/v1/nodes": dial tcp 34.170.211.246:443: connect: connection refused W0118 02:32:09.520686 5835 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.170.211.246/api/v1/nodes": dial tcp 34.170.211.246:443: connect: connection refused W0118 02:32:19.523201 5835 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.170.211.246/api/v1/nodes": dial tcp 34.170.211.246:443: connect: connection refused W0118 02:32:29.536299 5835 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.170.211.246/api/v1/nodes": dial tcp 34.170.211.246:443: connect: connection refused W0118 02:32:39.543106 5835 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.170.211.246/api/v1/nodes": dial tcp 34.170.211.246:443: connect: connection refused W0118 02:32:49.545776 5835 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.170.211.246/api/v1/nodes": dial tcp 34.170.211.246:443: connect: connection refused W0118 02:32:59.548814 5835 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.170.211.246/api/v1/nodes": dial tcp 34.170.211.246:443: connect: connection refused W0118 02:33:09.552062 5835 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.170.211.246/api/v1/nodes": dial tcp 34.170.211.246:443: connect: connection refused W0118 02:33:19.554958 5835 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.170.211.246/api/v1/nodes": dial tcp 34.170.211.246:443: connect: connection refused W0118 02:33:39.558310 5835 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.170.211.246/api/v1/nodes": net/http: TLS handshake timeout W0118 02:33:59.561353 5835 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.170.211.246/api/v1/nodes": net/http: TLS handshake timeout I0118 02:34:09.967430 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 6 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-4d3f machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-4d3f" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-8rwg machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-8rwg" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-dfz0 machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-dfz0" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-g1g5 machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-g1g5" has not yet joined cluster Pod kube-system/kube-controller-manager-control-plane-us-central1-a-kc2d system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-a-kc2d" is pending Validation Failed W0118 02:34:10.587667 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:34:18.489795 5471 boskos.go:86] Sending heartbeat to Boskos I0118 02:34:20.949293 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-4d3f machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-4d3f" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-8rwg machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-8rwg" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-dfz0 machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-dfz0" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-g1g5 machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-g1g5" has not yet joined cluster Pod kube-system/kube-controller-manager-control-plane-us-central1-a-kc2d system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-a-kc2d" is not ready (kube-controller-manager) Validation Failed W0118 02:34:21.515836 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:34:31.938167 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-4d3f machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-4d3f" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-8rwg machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-8rwg" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-dfz0 machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-dfz0" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-g1g5 machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-g1g5" has not yet joined cluster Pod kube-system/kube-controller-manager-control-plane-us-central1-a-kc2d system-cluster-critical pod "kube-controller-manager-control-plane-us-central1-a-kc2d" is not ready (kube-controller-manager) Validation Failed W0118 02:34:32.477712 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:34:42.854434 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 6 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/control-plane-us-central1-a-kc2d machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/control-plane-us-central1-a-kc2d" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-4d3f machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-4d3f" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-8rwg machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-8rwg" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-dfz0 machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-dfz0" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-g1g5 machine "https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-central1-a/instances/nodes-us-central1-a-g1g5" has not yet joined cluster Validation Failed W0118 02:34:43.363929 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:34:53.769765 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 12 lines ... Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is pending Pod kube-system/cloud-controller-manager-7kg8z system-cluster-critical pod "cloud-controller-manager-7kg8z" is pending Pod kube-system/coredns-7bc44848df-vb8qw system-cluster-critical pod "coredns-7bc44848df-vb8qw" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-bnxw7 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-bnxw7" is pending Pod kube-system/kube-scheduler-control-plane-us-central1-a-kc2d system-cluster-critical pod "kube-scheduler-control-plane-us-central1-a-kc2d" is pending Validation Failed W0118 02:34:54.300810 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:35:04.691906 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 12 lines ... Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is pending Pod kube-system/cloud-controller-manager-7kg8z system-cluster-critical pod "cloud-controller-manager-7kg8z" is pending Pod kube-system/coredns-7bc44848df-vb8qw system-cluster-critical pod "coredns-7bc44848df-vb8qw" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-bnxw7 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-bnxw7" is pending Pod kube-system/etcd-manager-events-control-plane-us-central1-a-kc2d system-cluster-critical pod "etcd-manager-events-control-plane-us-central1-a-kc2d" is pending Validation Failed W0118 02:35:05.217052 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:35:15.629370 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 14 lines ... Pod kube-system/calico-node-mw9g7 system-node-critical pod "calico-node-mw9g7" is pending Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is pending Pod kube-system/calico-node-xkw4r system-node-critical pod "calico-node-xkw4r" is pending Pod kube-system/coredns-7bc44848df-vb8qw system-cluster-critical pod "coredns-7bc44848df-vb8qw" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-bnxw7 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-bnxw7" is pending Validation Failed W0118 02:35:16.241682 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:35:26.642612 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 19 lines ... Pod kube-system/calico-node-mw9g7 system-node-critical pod "calico-node-mw9g7" is pending Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Pod kube-system/calico-node-xkw4r system-node-critical pod "calico-node-xkw4r" is pending Pod kube-system/coredns-7bc44848df-vb8qw system-cluster-critical pod "coredns-7bc44848df-vb8qw" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-bnxw7 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-bnxw7" is pending Validation Failed W0118 02:35:27.266252 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:35:37.626289 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 12 lines ... Pod kube-system/calico-node-gsbnl system-node-critical pod "calico-node-gsbnl" is pending Pod kube-system/calico-node-mw9g7 system-node-critical pod "calico-node-mw9g7" is pending Pod kube-system/calico-node-xkw4r system-node-critical pod "calico-node-xkw4r" is pending Pod kube-system/coredns-7bc44848df-vb8qw system-cluster-critical pod "coredns-7bc44848df-vb8qw" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-bnxw7 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-bnxw7" is pending Validation Failed W0118 02:35:38.213959 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:35:48.608565 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 11 lines ... Pod kube-system/calico-node-gnmsn system-node-critical pod "calico-node-gnmsn" is not ready (calico-node) Pod kube-system/calico-node-gsbnl system-node-critical pod "calico-node-gsbnl" is not ready (calico-node) Pod kube-system/calico-node-mw9g7 system-node-critical pod "calico-node-mw9g7" is not ready (calico-node) Pod kube-system/calico-node-xkw4r system-node-critical pod "calico-node-xkw4r" is not ready (calico-node) Pod kube-system/coredns-7bc44848df-6jr6d system-cluster-critical pod "coredns-7bc44848df-6jr6d" is pending Validation Failed W0118 02:35:49.183119 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:35:59.636097 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 10 lines ... KIND NAME MESSAGE Pod kube-system/calico-node-gnmsn system-node-critical pod "calico-node-gnmsn" is not ready (calico-node) Pod kube-system/calico-node-gsbnl system-node-critical pod "calico-node-gsbnl" is not ready (calico-node) Pod kube-system/calico-node-mw9g7 system-node-critical pod "calico-node-mw9g7" is not ready (calico-node) Pod kube-system/calico-node-xkw4r system-node-critical pod "calico-node-xkw4r" is not ready (calico-node) Validation Failed W0118 02:36:00.258899 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:36:10.615932 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 11 lines ... Pod kube-system/calico-node-gnmsn system-node-critical pod "calico-node-gnmsn" is not ready (calico-node) Pod kube-system/calico-node-gsbnl system-node-critical pod "calico-node-gsbnl" is not ready (calico-node) Pod kube-system/calico-node-mw9g7 system-node-critical pod "calico-node-mw9g7" is not ready (calico-node) Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Pod kube-system/calico-node-xkw4r system-node-critical pod "calico-node-xkw4r" is not ready (calico-node) Validation Failed W0118 02:36:11.176760 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:36:21.629310 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:36:22.171565 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:36:32.544985 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:36:33.123980 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:36:43.519400 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:36:44.086777 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:36:54.465031 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:36:55.189293 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:37:05.680438 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:37:06.205399 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:37:16.612388 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:37:17.181900 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:37:27.635656 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:37:28.138961 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:37:38.526439 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:37:39.080078 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:37:49.473786 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:37:50.005780 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:38:00.474444 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:38:01.079940 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:38:11.487096 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:38:12.103070 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:38:22.507006 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:38:23.090184 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:38:33.632295 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:38:34.244635 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:38:44.673036 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:38:45.193315 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:38:55.642361 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:38:56.295740 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:39:06.688288 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:39:07.278109 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:39:17.750553 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:39:18.277894 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:39:18.498679 5471 boskos.go:86] Sending heartbeat to Boskos I0118 02:39:28.723796 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 8 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:39:29.243847 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:39:39.658066 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:39:40.211233 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:39:50.598047 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:39:51.122994 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:40:01.519086 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:40:02.061960 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:40:12.405399 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:40:12.950147 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:40:23.491852 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:40:24.018896 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:40:34.469231 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:40:35.081546 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:40:45.501267 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:40:46.087855 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:40:56.478316 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:40:57.036185 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:41:07.530539 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:41:08.051874 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:41:18.497640 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:41:19.085456 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:41:29.496696 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:41:30.044351 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:41:40.447060 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:41:41.044115 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:41:51.451523 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:41:51.995950 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:42:02.460791 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:42:03.025021 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:42:13.498472 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:42:13.999405 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:42:24.441354 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:42:25.015303 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:42:35.569048 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:42:36.224781 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:42:46.739957 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:42:47.268556 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:42:57.647140 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:42:58.220718 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:43:08.631236 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:43:09.189106 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:43:19.575579 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:43:20.237731 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:43:30.609950 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:43:31.152573 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:43:41.606647 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:43:42.151711 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:43:52.557212 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:43:53.155787 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:44:03.641263 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:44:04.208965 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:44:14.622181 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:44:15.173917 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:44:18.507586 5471 boskos.go:86] Sending heartbeat to Boskos I0118 02:44:25.548515 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 8 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:44:26.151629 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:44:36.585765 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:44:37.090458 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:44:47.539268 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:44:48.065210 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:44:58.513637 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:44:59.060654 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:45:09.446827 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:45:09.970672 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:45:20.409212 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:45:20.942982 5835 validate_cluster.go:232] (will retry): cluster not yet healthy I0118 02:45:31.304282 5835 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-g1g5 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-tndzk system-node-critical pod "calico-node-tndzk" is not ready (calico-node) Validation Failed W0118 02:45:31.804789 5835 validate_cluster.go:232] (will retry): cluster not yet healthy Error: validation failed: wait time exceeded during validation I0118 02:45:41.813693 5471 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/99e826ad-96d7-11ed-824d-f64c9135b4ea/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh299717887/key --ssh-user prow I0118 02:45:41.813755 5471 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/99e826ad-96d7-11ed-824d-f64c9135b4ea/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh299717887/key --ssh-user prow I0118 02:45:42.241324 5845 gce.go:99] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] 2023/01/18 02:45:45 Dumping node control-plane-us-central1-a-kc2d 2023/01/18 02:45:48 error dumping node control-plane-us-central1-a-kc2d: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/01/18 02:45:48 error dumping node control-plane-us-central1-a-kc2d: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/01/18 02:45:48 Dumping node nodes-us-central1-a-4d3f 2023/01/18 02:45:50 Dumping node nodes-us-central1-a-8rwg 2023/01/18 02:45:52 Dumping node nodes-us-central1-a-dfz0 2023/01/18 02:45:54 Dumping node nodes-us-central1-a-g1g5 I0118 02:45:56.912805 5471 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/99e826ad-96d7-11ed-824d-f64c9135b4ea/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local -o yaml I0118 02:45:56.912839 5471 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/99e826ad-96d7-11ed-824d-f64c9135b4ea/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local -o yaml ... skipping 154 lines ... Deleted cluster: "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" I0118 02:48:02.003552 5471 gcs.go:88] gsutil -u k8s-jkns-gci-gce-sd-log rm -r gs://k8s-jkns-gci-gce-sd-log-state-99 I0118 02:48:02.003602 5471 local.go:42] ⚙️ gsutil -u k8s-jkns-gci-gce-sd-log rm -r gs://k8s-jkns-gci-gce-sd-log-state-99 Removing gs://k8s-jkns-gci-gce-sd-log-state-99/... I0118 02:48:04.434727 5471 down.go:62] releasing boskos project I0118 02:48:04.443788 5471 boskos.go:83] Boskos heartbeat func received signal to close Error: exit status 1 + EXIT_VALUE=1 + set +o xtrace