e2e-kops-grid-gce-calico-u2004-k26 #1619280962087030784
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 1 failed / 1 succeeded |
| Started | |
| Elapsed | 20m2s |
| Revision | master |
exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest2 Down
... skipping 163 lines ... ............................................Updated [https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew]. ..done. WARNING: No host aliases were added to your SSH configs because you do not have any running instances. Try running this command again after running some instances. I0128 10:29:29.391860 5549 up.go:44] Cleaning up any leaked resources from previous cluster I0128 10:29:29.392022 5549 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/58d629fd-9ef6-11ed-9a0c-f25a29267a91/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh63528120/key --ssh-user prow I0128 10:29:29.392040 5549 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/58d629fd-9ef6-11ed-9a0c-f25a29267a91/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh63528120/key --ssh-user prow Error: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" not found W0128 10:29:29.649072 5549 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0128 10:29:29.649119 5549 down.go:48] /home/prow/go/src/k8s.io/kops/_rundir/58d629fd-9ef6-11ed-9a0c-f25a29267a91/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --yes I0128 10:29:29.649137 5549 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/58d629fd-9ef6-11ed-9a0c-f25a29267a91/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --yes Error: error reading cluster configuration: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" not found I0128 10:29:29.817159 5549 gcs.go:51] gsutil ls -b -p k8s-gce-gci-mstr-1-6-ctl-skew gs://k8s-gce-gci-mstr-1-6-ctl-skew-state-58 I0128 10:29:29.817202 5549 local.go:42] ⚙️ gsutil ls -b -p k8s-gce-gci-mstr-1-6-ctl-skew gs://k8s-gce-gci-mstr-1-6-ctl-skew-state-58 I0128 10:29:31.757087 5549 gcs.go:70] gsutil mb -p k8s-gce-gci-mstr-1-6-ctl-skew gs://k8s-gce-gci-mstr-1-6-ctl-skew-state-58 I0128 10:29:31.757118 5549 local.go:42] ⚙️ gsutil mb -p k8s-gce-gci-mstr-1-6-ctl-skew gs://k8s-gce-gci-mstr-1-6-ctl-skew-state-58 Creating gs://k8s-gce-gci-mstr-1-6-ctl-skew-state-58/... I0128 10:29:34.202823 5549 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2023/01/28 10:29:34 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I0128 10:29:34.215218 5549 http.go:37] curl https://ip.jsb.workers.dev I0128 10:29:34.304141 5549 up.go:167] /home/prow/go/src/k8s.io/kops/_rundir/58d629fd-9ef6-11ed-9a0c-f25a29267a91/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.26.1 --ssh-public-key /tmp/kops-ssh63528120/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 34.121.55.31/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-east4-a --master-size e2-standard-2 --project k8s-gce-gci-mstr-1-6-ctl-skew I0128 10:29:34.304185 5549 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/58d629fd-9ef6-11ed-9a0c-f25a29267a91/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.26.1 --ssh-public-key /tmp/kops-ssh63528120/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 34.121.55.31/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-east4-a --master-size e2-standard-2 --project k8s-gce-gci-mstr-1-6-ctl-skew I0128 10:29:34.354158 5873 create_cluster.go:882] Using SSH public key: /tmp/kops-ssh63528120/key.pub I0128 10:29:34.658711 5873 new_cluster.go:573] VMs will be configured to use specified Service Account: default I0128 10:29:34.658858 5873 new_cluster.go:1338] Cloud Provider ID: "gce" ... skipping 349 lines ... I0128 10:29:42.881469 5893 keypair.go:226] Issuing new certificate: "etcd-peers-ca-events" I0128 10:29:42.885476 5893 keypair.go:226] Issuing new certificate: "etcd-manager-ca-main" W0128 10:29:43.060538 5893 vfs_keystorereader.go:143] CA private key was not found I0128 10:29:43.063262 5893 keypair.go:226] Issuing new certificate: "service-account" I0128 10:29:43.171601 5893 keypair.go:226] Issuing new certificate: "kubernetes-ca" I0128 10:29:53.321192 5893 executor.go:111] Tasks: 42 done / 67 total; 19 can run W0128 10:30:06.827489 5893 executor.go:139] error running task "ForwardingRule/api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local" (9m46s remaining to succeed): error creating ForwardingRule "api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local": googleapi: Error 400: The resource 'projects/k8s-gce-gci-mstr-1-6-ctl-skew/regions/us-east4/targetPools/api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local' is not ready, resourceNotReady I0128 10:30:06.827553 5893 executor.go:111] Tasks: 60 done / 67 total; 5 can run I0128 10:30:13.753987 5893 executor.go:111] Tasks: 65 done / 67 total; 2 can run I0128 10:30:27.727665 5893 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0128 10:30:27.795485 5893 update_cluster.go:323] Exporting kubeconfig for cluster kOps has set your kubectl context to e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local ... skipping 7 lines ... * read about installing addons at: https://kops.sigs.k8s.io/addons. I0128 10:30:38.160209 5549 up.go:251] /home/prow/go/src/k8s.io/kops/_rundir/58d629fd-9ef6-11ed-9a0c-f25a29267a91/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --count 10 --wait 15m0s I0128 10:30:38.160269 5549 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/58d629fd-9ef6-11ed-9a0c-f25a29267a91/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --count 10 --wait 15m0s Validating cluster e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local W0128 10:31:08.524225 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": dial tcp 34.85.231.25:443: i/o timeout W0128 10:31:18.551678 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": dial tcp 34.85.231.25:443: connect: connection refused W0128 10:31:28.582572 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": dial tcp 34.85.231.25:443: connect: connection refused W0128 10:31:38.610492 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": dial tcp 34.85.231.25:443: connect: connection refused W0128 10:31:48.638414 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": dial tcp 34.85.231.25:443: connect: connection refused W0128 10:31:58.665275 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": dial tcp 34.85.231.25:443: connect: connection refused W0128 10:32:08.692737 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": dial tcp 34.85.231.25:443: connect: connection refused W0128 10:32:18.719279 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": dial tcp 34.85.231.25:443: connect: connection refused W0128 10:32:28.746383 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": dial tcp 34.85.231.25:443: connect: connection refused W0128 10:32:38.774121 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": dial tcp 34.85.231.25:443: connect: connection refused W0128 10:32:58.803681 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": net/http: TLS handshake timeout W0128 10:33:18.832759 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": net/http: TLS handshake timeout W0128 10:33:28.858879 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": dial tcp 34.85.231.25:443: connect: connection refused W0128 10:33:38.887763 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": dial tcp 34.85.231.25:443: connect: connection refused W0128 10:33:58.914318 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": net/http: TLS handshake timeout W0128 10:34:08.940170 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": dial tcp 34.85.231.25:443: connect: connection refused I0128 10:34:16.923495 5549 boskos.go:86] Sending heartbeat to Boskos W0128 10:34:18.966772 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": dial tcp 34.85.231.25:443: connect: connection refused W0128 10:34:28.992731 5912 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.85.231.25/api/v1/nodes": dial tcp 34.85.231.25:443: connect: connection refused I0128 10:34:41.420409 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 5 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/control-plane-us-east4-a-k8f5 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/control-plane-us-east4-a-k8f5" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-3r29 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-3r29" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-40v8 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-40v8" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-slfh machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-slfh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-z5b3 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-z5b3" has not yet joined cluster Validation Failed W0128 10:34:42.089511 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:34:52.515952 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 6 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/control-plane-us-east4-a-k8f5 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/control-plane-us-east4-a-k8f5" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-3r29 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-3r29" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-40v8 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-40v8" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-slfh machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-slfh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-z5b3 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-z5b3" has not yet joined cluster Validation Failed W0128 10:34:53.110110 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:35:03.435954 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-3r29 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-3r29" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-40v8 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-40v8" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-slfh machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-slfh" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-z5b3 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-z5b3" has not yet joined cluster Pod kube-system/etcd-manager-events-control-plane-us-east4-a-k8f5 system-cluster-critical pod "etcd-manager-events-control-plane-us-east4-a-k8f5" is pending Validation Failed W0128 10:35:04.193418 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:35:14.532744 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 10 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-z5b3 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-mstr-1-6-ctl-skew/zones/us-east4-a/instances/nodes-us-east4-a-z5b3" has not yet joined cluster Pod kube-system/calico-kube-controllers-55f488656-b2pkd system-cluster-critical pod "calico-kube-controllers-55f488656-b2pkd" is pending Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is pending Pod kube-system/dns-controller-6d5f4bf5d6-bgvxl system-cluster-critical pod "dns-controller-6d5f4bf5d6-bgvxl" is pending Pod kube-system/kube-scheduler-control-plane-us-east4-a-k8f5 system-cluster-critical pod "kube-scheduler-control-plane-us-east4-a-k8f5" is pending Validation Failed W0128 10:35:15.270689 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:35:25.602129 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 13 lines ... Pod kube-system/cloud-controller-manager-8qg96 system-cluster-critical pod "cloud-controller-manager-8qg96" is pending Pod kube-system/coredns-7bc44848df-bczlz system-cluster-critical pod "coredns-7bc44848df-bczlz" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-4s2sj system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-4s2sj" is pending Pod kube-system/dns-controller-6d5f4bf5d6-bgvxl system-cluster-critical pod "dns-controller-6d5f4bf5d6-bgvxl" is pending Pod kube-system/kops-controller-fd9cm system-cluster-critical pod "kops-controller-fd9cm" is pending Validation Failed W0128 10:35:26.330807 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:35:36.714036 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 14 lines ... Pod kube-system/coredns-7bc44848df-bczlz system-cluster-critical pod "coredns-7bc44848df-bczlz" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-4s2sj system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-4s2sj" is pending Pod kube-system/dns-controller-6d5f4bf5d6-bgvxl system-cluster-critical pod "dns-controller-6d5f4bf5d6-bgvxl" is pending Pod kube-system/etcd-manager-main-control-plane-us-east4-a-k8f5 system-cluster-critical pod "etcd-manager-main-control-plane-us-east4-a-k8f5" is pending Pod kube-system/kops-controller-fd9cm system-cluster-critical pod "kops-controller-fd9cm" is pending Validation Failed W0128 10:35:37.213545 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:35:47.627625 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 12 lines ... Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is pending Pod kube-system/cloud-controller-manager-8qg96 system-cluster-critical pod "cloud-controller-manager-8qg96" is pending Pod kube-system/coredns-7bc44848df-bczlz system-cluster-critical pod "coredns-7bc44848df-bczlz" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-4s2sj system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-4s2sj" is pending Pod kube-system/kube-apiserver-control-plane-us-east4-a-k8f5 system-cluster-critical pod "kube-apiserver-control-plane-us-east4-a-k8f5" is pending Validation Failed W0128 10:35:48.114273 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:35:58.577888 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 15 lines ... Pod kube-system/calico-node-f5qks system-node-critical pod "calico-node-f5qks" is pending Pod kube-system/calico-node-vndvr system-node-critical pod "calico-node-vndvr" is pending Pod kube-system/coredns-7bc44848df-bczlz system-cluster-critical pod "coredns-7bc44848df-bczlz" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-4s2sj system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-4s2sj" is pending Pod kube-system/kube-proxy-nodes-us-east4-a-z5b3 system-node-critical pod "kube-proxy-nodes-us-east4-a-z5b3" is pending Validation Failed W0128 10:35:59.273403 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:36:09.601778 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 15 lines ... Pod kube-system/calico-node-7gdjq system-node-critical pod "calico-node-7gdjq" is pending Pod kube-system/calico-node-f5qks system-node-critical pod "calico-node-f5qks" is pending Pod kube-system/calico-node-vndvr system-node-critical pod "calico-node-vndvr" is pending Pod kube-system/coredns-7bc44848df-bczlz system-cluster-critical pod "coredns-7bc44848df-bczlz" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-4s2sj system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-4s2sj" is pending Validation Failed W0128 10:36:10.131644 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:36:20.447382 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 12 lines ... Pod kube-system/calico-node-7gdjq system-node-critical pod "calico-node-7gdjq" is not ready (calico-node) Pod kube-system/calico-node-f5qks system-node-critical pod "calico-node-f5qks" is pending Pod kube-system/calico-node-vndvr system-node-critical pod "calico-node-vndvr" is pending Pod kube-system/coredns-7bc44848df-bczlz system-cluster-critical pod "coredns-7bc44848df-bczlz" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-4s2sj system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-4s2sj" is pending Validation Failed W0128 10:36:21.257724 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:36:31.651553 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 10 lines ... KIND NAME MESSAGE Pod kube-system/calico-node-2h6sd system-node-critical pod "calico-node-2h6sd" is not ready (calico-node) Pod kube-system/calico-node-7gdjq system-node-critical pod "calico-node-7gdjq" is not ready (calico-node) Pod kube-system/calico-node-f5qks system-node-critical pod "calico-node-f5qks" is not ready (calico-node) Pod kube-system/calico-node-vndvr system-node-critical pod "calico-node-vndvr" is not ready (calico-node) Validation Failed W0128 10:36:32.157548 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:36:42.549751 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 10 lines ... KIND NAME MESSAGE Pod kube-system/calico-node-2h6sd system-node-critical pod "calico-node-2h6sd" is not ready (calico-node) Pod kube-system/calico-node-7gdjq system-node-critical pod "calico-node-7gdjq" is not ready (calico-node) Pod kube-system/calico-node-f5qks system-node-critical pod "calico-node-f5qks" is not ready (calico-node) Pod kube-system/calico-node-vndvr system-node-critical pod "calico-node-vndvr" is not ready (calico-node) Validation Failed W0128 10:36:43.107706 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:36:53.505668 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 9 lines ... VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-2h6sd system-node-critical pod "calico-node-2h6sd" is not ready (calico-node) Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Pod kube-system/calico-node-f5qks system-node-critical pod "calico-node-f5qks" is not ready (calico-node) Validation Failed W0128 10:36:54.089113 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:37:04.453592 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:37:04.922359 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:37:15.366302 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:37:15.854202 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:37:26.271116 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:37:26.725260 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:37:37.069809 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:37:37.583353 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:37:47.938545 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:37:48.525848 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:37:58.903182 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:37:59.408096 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:38:09.714580 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:38:10.303684 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:38:20.636067 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:38:21.197001 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:38:31.682281 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:38:32.211441 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:38:42.602527 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:38:43.186686 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:38:53.570618 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:38:54.021988 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:39:04.380688 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:39:04.813377 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:39:15.138486 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:39:15.725450 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:39:16.933480 5549 boskos.go:86] Sending heartbeat to Boskos I0128 10:39:26.050873 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 ... skipping 8 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:39:26.500397 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:39:36.828930 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:39:37.425026 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:39:47.762803 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:39:48.261548 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:39:58.701901 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:39:59.486680 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:40:09.878260 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:40:10.460985 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:40:20.799087 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:40:21.248595 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:40:31.551081 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:40:32.093819 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:40:42.408505 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:40:42.869804 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:40:53.270438 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:40:53.987115 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:41:04.367292 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:41:04.838499 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:41:15.195200 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:41:15.693743 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:41:26.062157 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:41:26.674873 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:41:36.989741 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:41:37.449892 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:41:47.803100 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:41:48.379479 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:41:58.842899 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:41:59.562018 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:42:09.873340 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:42:10.315460 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:42:20.762837 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:42:21.240033 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:42:31.652940 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:42:32.206081 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:42:42.537085 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:42:42.997583 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:42:53.343226 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:42:53.932817 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:43:04.377933 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:43:04.860390 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:43:15.185827 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:43:15.706569 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:43:26.241289 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:43:26.834080 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:43:37.206590 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:43:37.692248 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:43:48.027773 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:43:48.514232 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:43:58.900165 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:43:59.410628 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:44:09.779236 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:44:10.229178 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:44:16.945239 5549 boskos.go:86] Sending heartbeat to Boskos I0128 10:44:20.565546 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 ... skipping 8 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:44:21.012777 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:44:31.306179 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:44:31.800441 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:44:42.163709 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:44:42.655068 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:44:53.025302 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:44:53.523908 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:45:04.054367 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:45:04.541971 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:45:14.863516 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:45:15.380429 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:45:25.718410 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:45:26.364867 5912 validate_cluster.go:232] (will retry): cluster not yet healthy I0128 10:45:36.755548 5912 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-z5b3 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-662b6 system-node-critical pod "calico-node-662b6" is not ready (calico-node) Validation Failed W0128 10:45:37.361567 5912 validate_cluster.go:232] (will retry): cluster not yet healthy Error: validation failed: wait time exceeded during validation I0128 10:45:47.374708 5549 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/58d629fd-9ef6-11ed-9a0c-f25a29267a91/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh63528120/key --ssh-user prow I0128 10:45:47.374757 5549 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/58d629fd-9ef6-11ed-9a0c-f25a29267a91/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh63528120/key --ssh-user prow I0128 10:45:47.864610 5922 gce.go:99] Scanning zones: [us-east4-c us-east4-b us-east4-a] 2023/01/28 10:45:52 Dumping node control-plane-us-east4-a-k8f5 2023/01/28 10:45:57 error dumping node control-plane-us-east4-a-k8f5: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/01/28 10:45:57 error dumping node control-plane-us-east4-a-k8f5: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/01/28 10:45:57 Dumping node nodes-us-east4-a-3r29 2023/01/28 10:46:00 Dumping node nodes-us-east4-a-40v8 2023/01/28 10:46:02 Dumping node nodes-us-east4-a-slfh 2023/01/28 10:46:05 Dumping node nodes-us-east4-a-z5b3 I0128 10:46:08.126414 5549 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/58d629fd-9ef6-11ed-9a0c-f25a29267a91/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local -o yaml I0128 10:46:08.126451 5549 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/58d629fd-9ef6-11ed-9a0c-f25a29267a91/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local -o yaml ... skipping 154 lines ... Deleted cluster: "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" I0128 10:48:11.140526 5549 gcs.go:88] gsutil -u k8s-gce-gci-mstr-1-6-ctl-skew rm -r gs://k8s-gce-gci-mstr-1-6-ctl-skew-state-58 I0128 10:48:11.140580 5549 local.go:42] ⚙️ gsutil -u k8s-gce-gci-mstr-1-6-ctl-skew rm -r gs://k8s-gce-gci-mstr-1-6-ctl-skew-state-58 Removing gs://k8s-gce-gci-mstr-1-6-ctl-skew-state-58/... I0128 10:48:13.650567 5549 down.go:62] releasing boskos project I0128 10:48:13.659276 5549 boskos.go:83] Boskos heartbeat func received signal to close Error: exit status 1 + EXIT_VALUE=1 + set +o xtrace