e2e-kops-grid-gce-calico-u2004-k26 #1616502670133039104
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 1 failed / 1 succeeded |
| Started | |
| Elapsed | 20m0s |
| Revision | master |
exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest2 Down
... skipping 163 lines ... ................................Updated [https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5]. .done. WARNING: No host aliases were added to your SSH configs because you do not have any running instances. Try running this command again after running some instances. I0120 18:29:22.818543 5509 up.go:44] Cleaning up any leaked resources from previous cluster I0120 18:29:22.818660 5509 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/15cd466b-98f0-11ed-824d-f64c9135b4ea/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh468301733/key --ssh-user prow I0120 18:29:22.818686 5509 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/15cd466b-98f0-11ed-824d-f64c9135b4ea/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh468301733/key --ssh-user prow Error: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" not found W0120 18:29:23.034640 5509 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0120 18:29:23.034713 5509 down.go:48] /home/prow/go/src/k8s.io/kops/_rundir/15cd466b-98f0-11ed-824d-f64c9135b4ea/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --yes I0120 18:29:23.034741 5509 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/15cd466b-98f0-11ed-824d-f64c9135b4ea/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --yes Error: error reading cluster configuration: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" not found I0120 18:29:23.163425 5509 gcs.go:51] gsutil ls -b -p e2e-gce-gci-ci-serial-1-5 gs://e2e-gce-gci-ci-serial-1-5-state-15 I0120 18:29:23.163475 5509 local.go:42] ⚙️ gsutil ls -b -p e2e-gce-gci-ci-serial-1-5 gs://e2e-gce-gci-ci-serial-1-5-state-15 I0120 18:29:25.051423 5509 gcs.go:70] gsutil mb -p e2e-gce-gci-ci-serial-1-5 gs://e2e-gce-gci-ci-serial-1-5-state-15 I0120 18:29:25.051456 5509 local.go:42] ⚙️ gsutil mb -p e2e-gce-gci-ci-serial-1-5 gs://e2e-gce-gci-ci-serial-1-5-state-15 Creating gs://e2e-gce-gci-ci-serial-1-5-state-15/... I0120 18:29:27.533780 5509 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2023/01/20 18:29:27 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I0120 18:29:27.545504 5509 http.go:37] curl https://ip.jsb.workers.dev I0120 18:29:27.645147 5509 up.go:167] /home/prow/go/src/k8s.io/kops/_rundir/15cd466b-98f0-11ed-824d-f64c9135b4ea/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.26.1 --ssh-public-key /tmp/kops-ssh468301733/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 34.68.176.126/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-west3-a --master-size e2-standard-2 --project e2e-gce-gci-ci-serial-1-5 I0120 18:29:27.645193 5509 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/15cd466b-98f0-11ed-824d-f64c9135b4ea/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.26.1 --ssh-public-key /tmp/kops-ssh468301733/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 34.68.176.126/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-west3-a --master-size e2-standard-2 --project e2e-gce-gci-ci-serial-1-5 I0120 18:29:27.692087 5831 create_cluster.go:882] Using SSH public key: /tmp/kops-ssh468301733/key.pub I0120 18:29:27.945252 5831 new_cluster.go:573] VMs will be configured to use specified Service Account: default I0120 18:29:27.945376 5831 new_cluster.go:1338] Cloud Provider ID: "gce" ... skipping 349 lines ... I0120 18:29:36.385053 5849 keypair.go:226] Issuing new certificate: "etcd-peers-ca-events" I0120 18:29:36.393736 5849 keypair.go:226] Issuing new certificate: "etcd-peers-ca-main" W0120 18:29:36.470234 5849 vfs_keystorereader.go:143] CA private key was not found I0120 18:29:36.568482 5849 keypair.go:226] Issuing new certificate: "kubernetes-ca" I0120 18:29:36.678553 5849 keypair.go:226] Issuing new certificate: "service-account" I0120 18:29:49.037054 5849 executor.go:111] Tasks: 42 done / 67 total; 19 can run W0120 18:30:02.858431 5849 executor.go:139] error running task "ForwardingRule/api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local" (9m46s remaining to succeed): error creating ForwardingRule "api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local": googleapi: Error 400: The resource 'projects/e2e-gce-gci-ci-serial-1-5/regions/us-west3/targetPools/api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local' is not ready, resourceNotReady I0120 18:30:02.858487 5849 executor.go:111] Tasks: 60 done / 67 total; 5 can run I0120 18:30:09.858898 5849 executor.go:111] Tasks: 65 done / 67 total; 2 can run I0120 18:30:22.900097 5849 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0120 18:30:23.044147 5849 update_cluster.go:323] Exporting kubeconfig for cluster kOps has set your kubectl context to e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local ... skipping 7 lines ... * read about installing addons at: https://kops.sigs.k8s.io/addons. I0120 18:30:33.531042 5509 up.go:251] /home/prow/go/src/k8s.io/kops/_rundir/15cd466b-98f0-11ed-824d-f64c9135b4ea/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --count 10 --wait 15m0s I0120 18:30:33.531093 5509 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/15cd466b-98f0-11ed-824d-f64c9135b4ea/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --count 10 --wait 15m0s Validating cluster e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local W0120 18:31:03.929414 5870 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.106.39.141/api/v1/nodes": dial tcp 34.106.39.141:443: i/o timeout W0120 18:31:13.953278 5870 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.106.39.141/api/v1/nodes": dial tcp 34.106.39.141:443: connect: connection refused W0120 18:31:23.975764 5870 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.106.39.141/api/v1/nodes": dial tcp 34.106.39.141:443: connect: connection refused W0120 18:31:33.998918 5870 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.106.39.141/api/v1/nodes": dial tcp 34.106.39.141:443: connect: connection refused W0120 18:31:44.021263 5870 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.106.39.141/api/v1/nodes": dial tcp 34.106.39.141:443: connect: connection refused W0120 18:31:54.043281 5870 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.106.39.141/api/v1/nodes": dial tcp 34.106.39.141:443: connect: connection refused W0120 18:32:04.065873 5870 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.106.39.141/api/v1/nodes": dial tcp 34.106.39.141:443: connect: connection refused W0120 18:32:14.088319 5870 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.106.39.141/api/v1/nodes": dial tcp 34.106.39.141:443: connect: connection refused W0120 18:32:24.110442 5870 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.106.39.141/api/v1/nodes": dial tcp 34.106.39.141:443: connect: connection refused W0120 18:32:34.135890 5870 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.106.39.141/api/v1/nodes": dial tcp 34.106.39.141:443: connect: connection refused W0120 18:32:44.157717 5870 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.106.39.141/api/v1/nodes": dial tcp 34.106.39.141:443: connect: connection refused W0120 18:33:04.180833 5870 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.106.39.141/api/v1/nodes": net/http: TLS handshake timeout W0120 18:33:24.206198 5870 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.106.39.141/api/v1/nodes": net/http: TLS handshake timeout I0120 18:33:34.719856 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 5 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/control-plane-us-west3-a-qf71 machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/control-plane-us-west3-a-qf71" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-g0dx machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-g0dx" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-jcwc machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-jcwc" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-lf69 machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-lf69" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-n183 machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-n183" has not yet joined cluster Validation Failed W0120 18:33:35.265146 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:33:45.670138 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-g0dx machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-g0dx" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-jcwc machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-jcwc" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-lf69 machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-lf69" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-n183 machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-n183" has not yet joined cluster Pod kube-system/kube-controller-manager-control-plane-us-west3-a-qf71 system-cluster-critical pod "kube-controller-manager-control-plane-us-west3-a-qf71" is not ready (kube-controller-manager) Validation Failed W0120 18:33:46.170741 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:33:56.678912 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 6 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/control-plane-us-west3-a-qf71 machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/control-plane-us-west3-a-qf71" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-g0dx machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-g0dx" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-jcwc machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-jcwc" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-lf69 machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-lf69" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-n183 machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-n183" has not yet joined cluster Validation Failed W0120 18:33:57.203242 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:34:07.686048 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-g0dx machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-g0dx" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-jcwc machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-jcwc" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-lf69 machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-lf69" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-n183 machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-n183" has not yet joined cluster Pod kube-system/kube-scheduler-control-plane-us-west3-a-qf71 system-cluster-critical pod "kube-scheduler-control-plane-us-west3-a-qf71" is pending Validation Failed W0120 18:34:08.200747 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:34:12.860379 5509 boskos.go:86] Sending heartbeat to Boskos I0120 18:34:18.651808 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 ... skipping 14 lines ... Pod kube-system/cloud-controller-manager-f67jp system-cluster-critical pod "cloud-controller-manager-f67jp" is pending Pod kube-system/coredns-7bc44848df-zv2nn system-cluster-critical pod "coredns-7bc44848df-zv2nn" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-bz4xj system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-bz4xj" is pending Pod kube-system/dns-controller-6d5f4bf5d6-8l5mm system-cluster-critical pod "dns-controller-6d5f4bf5d6-8l5mm" is pending Pod kube-system/kops-controller-pzrkw system-cluster-critical pod "kops-controller-pzrkw" is pending Validation Failed W0120 18:34:19.322520 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:34:29.694634 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 10 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-n183 machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-serial-1-5/zones/us-west3-a/instances/nodes-us-west3-a-n183" has not yet joined cluster Pod kube-system/calico-kube-controllers-55f488656-grjgn system-cluster-critical pod "calico-kube-controllers-55f488656-grjgn" is pending Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is pending Pod kube-system/coredns-7bc44848df-zv2nn system-cluster-critical pod "coredns-7bc44848df-zv2nn" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-bz4xj system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-bz4xj" is pending Validation Failed W0120 18:34:30.292988 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:34:40.692841 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 19 lines ... Pod kube-system/calico-node-jjpq7 system-node-critical pod "calico-node-jjpq7" is pending Pod kube-system/coredns-7bc44848df-zv2nn system-cluster-critical pod "coredns-7bc44848df-zv2nn" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-bz4xj system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-bz4xj" is pending Pod kube-system/kube-apiserver-control-plane-us-west3-a-qf71 system-cluster-critical pod "kube-apiserver-control-plane-us-west3-a-qf71" is pending Pod kube-system/kube-proxy-nodes-us-west3-a-lf69 system-node-critical pod "kube-proxy-nodes-us-west3-a-lf69" is pending Validation Failed W0120 18:34:41.249114 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:34:51.673107 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 17 lines ... Pod kube-system/calico-node-g5g5c system-node-critical pod "calico-node-g5g5c" is pending Pod kube-system/calico-node-gjb2k system-node-critical pod "calico-node-gjb2k" is pending Pod kube-system/calico-node-jjpq7 system-node-critical pod "calico-node-jjpq7" is pending Pod kube-system/coredns-7bc44848df-zv2nn system-cluster-critical pod "coredns-7bc44848df-zv2nn" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-bz4xj system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-bz4xj" is pending Validation Failed W0120 18:34:52.267747 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:35:02.683218 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 12 lines ... Pod kube-system/calico-node-g5g5c system-node-critical pod "calico-node-g5g5c" is pending Pod kube-system/calico-node-gjb2k system-node-critical pod "calico-node-gjb2k" is pending Pod kube-system/calico-node-jjpq7 system-node-critical pod "calico-node-jjpq7" is pending Pod kube-system/coredns-7bc44848df-zv2nn system-cluster-critical pod "coredns-7bc44848df-zv2nn" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-bz4xj system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-bz4xj" is pending Validation Failed W0120 18:35:03.235173 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:35:13.661608 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 12 lines ... Pod kube-system/calico-node-g5g5c system-node-critical pod "calico-node-g5g5c" is not ready (calico-node) Pod kube-system/calico-node-gjb2k system-node-critical pod "calico-node-gjb2k" is not ready (calico-node) Pod kube-system/calico-node-jjpq7 system-node-critical pod "calico-node-jjpq7" is not ready (calico-node) Pod kube-system/coredns-7bc44848df-zskz7 system-cluster-critical pod "coredns-7bc44848df-zskz7" is pending Pod kube-system/coredns-7bc44848df-zv2nn system-cluster-critical pod "coredns-7bc44848df-zv2nn" is pending Validation Failed W0120 18:35:14.158476 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:35:24.647348 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 10 lines ... KIND NAME MESSAGE Pod kube-system/calico-node-fxp4l system-node-critical pod "calico-node-fxp4l" is not ready (calico-node) Pod kube-system/calico-node-g5g5c system-node-critical pod "calico-node-g5g5c" is not ready (calico-node) Pod kube-system/calico-node-gjb2k system-node-critical pod "calico-node-gjb2k" is not ready (calico-node) Pod kube-system/calico-node-jjpq7 system-node-critical pod "calico-node-jjpq7" is not ready (calico-node) Validation Failed W0120 18:35:25.200188 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:35:35.662643 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 10 lines ... KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Pod kube-system/calico-node-fxp4l system-node-critical pod "calico-node-fxp4l" is not ready (calico-node) Pod kube-system/calico-node-g5g5c system-node-critical pod "calico-node-g5g5c" is not ready (calico-node) Pod kube-system/calico-node-gjb2k system-node-critical pod "calico-node-gjb2k" is not ready (calico-node) Validation Failed W0120 18:35:36.223911 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:35:46.660239 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:35:47.147483 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:35:57.593219 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:35:58.059934 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:36:08.497711 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:36:08.934585 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:36:19.396181 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:36:19.829026 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:36:30.223300 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:36:30.711538 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:36:41.116707 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:36:41.661811 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:36:52.111124 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:36:52.710027 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:37:03.121427 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:37:03.584043 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:37:14.008812 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:37:14.420615 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:37:24.855008 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:37:25.350849 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:37:35.774694 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:37:36.241197 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:37:46.730670 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:37:47.194333 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:37:57.729448 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:37:58.240805 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:38:08.708612 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:38:09.217142 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:38:19.662072 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:38:20.225347 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:38:30.663938 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:38:31.096052 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:38:41.568788 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:38:42.090477 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:38:52.529189 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:38:52.948742 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:39:03.402423 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:39:03.783738 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:39:12.872790 5509 boskos.go:86] Sending heartbeat to Boskos I0120 18:39:14.213382 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 ... skipping 8 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:39:14.631076 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:39:25.127593 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:39:25.582569 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:39:36.094543 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:39:36.601992 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:39:47.059863 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:39:47.520807 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:39:57.978028 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:39:58.401828 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:40:08.836073 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:40:09.400428 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:40:19.831542 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:40:20.222120 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:40:30.629666 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:40:31.215904 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:40:41.642376 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:40:42.138610 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:40:52.582515 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:40:53.137258 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:41:03.581541 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:41:04.023831 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:41:14.486399 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:41:14.946323 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:41:25.463118 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:41:25.907789 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:41:36.407043 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:41:36.849786 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:41:47.261284 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:41:47.658483 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:41:58.050448 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:41:58.513932 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:42:08.936774 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:42:09.361854 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:42:19.838047 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:42:20.281337 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:42:30.689652 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:42:31.153004 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:42:41.561712 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:42:42.018093 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:42:52.487865 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:42:52.900574 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:43:03.308007 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:43:03.773923 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:43:14.299375 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:43:14.723357 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:43:25.118853 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:43:25.577975 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:43:36.018074 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:43:36.487091 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:43:46.939934 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:43:47.516828 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:43:57.953193 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:43:58.463635 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:44:08.896487 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:44:09.322936 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:44:12.886498 5509 boskos.go:86] Sending heartbeat to Boskos I0120 18:44:19.741414 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 ... skipping 8 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:44:20.293006 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:44:30.742066 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:44:31.187807 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:44:41.591963 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:44:42.038468 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:44:52.430527 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:44:52.867772 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:45:03.281487 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:45:03.753666 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:45:14.185381 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:45:14.702424 5870 validate_cluster.go:232] (will retry): cluster not yet healthy I0120 18:45:25.116349 5870 gce_cloud.go:301] Scanning zones: [us-west3-a us-west3-b us-west3-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west3-a ControlPlane e2-standard-2 1 1 us-west3 nodes-us-west3-a Node e2-medium 4 4 us-west3 ... skipping 7 lines ... nodes-us-west3-a-n183 node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-bgghj system-node-critical pod "calico-node-bgghj" is not ready (calico-node) Validation Failed W0120 18:45:25.551968 5870 validate_cluster.go:232] (will retry): cluster not yet healthy Error: validation failed: wait time exceeded during validation I0120 18:45:35.558258 5509 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/15cd466b-98f0-11ed-824d-f64c9135b4ea/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh468301733/key --ssh-user prow I0120 18:45:35.558307 5509 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/15cd466b-98f0-11ed-824d-f64c9135b4ea/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh468301733/key --ssh-user prow I0120 18:45:35.978131 5881 gce.go:99] Scanning zones: [us-west3-a us-west3-b us-west3-c] 2023/01/20 18:45:40 Dumping node control-plane-us-west3-a-qf71 2023/01/20 18:45:43 error dumping node control-plane-us-west3-a-qf71: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/01/20 18:45:43 error dumping node control-plane-us-west3-a-qf71: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/01/20 18:45:43 Dumping node nodes-us-west3-a-g0dx 2023/01/20 18:45:46 Dumping node nodes-us-west3-a-jcwc 2023/01/20 18:45:48 Dumping node nodes-us-west3-a-lf69 2023/01/20 18:45:51 Dumping node nodes-us-west3-a-n183 I0120 18:45:53.802221 5509 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/15cd466b-98f0-11ed-824d-f64c9135b4ea/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local -o yaml I0120 18:45:53.802262 5509 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/15cd466b-98f0-11ed-824d-f64c9135b4ea/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local -o yaml ... skipping 154 lines ... Deleted cluster: "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" I0120 18:48:07.926773 5509 gcs.go:88] gsutil -u e2e-gce-gci-ci-serial-1-5 rm -r gs://e2e-gce-gci-ci-serial-1-5-state-15 I0120 18:48:07.927718 5509 local.go:42] ⚙️ gsutil -u e2e-gce-gci-ci-serial-1-5 rm -r gs://e2e-gce-gci-ci-serial-1-5-state-15 Removing gs://e2e-gce-gci-ci-serial-1-5-state-15/... I0120 18:48:10.744863 5509 down.go:62] releasing boskos project I0120 18:48:10.752813 5509 boskos.go:83] Boskos heartbeat func received signal to close Error: exit status 1 + EXIT_VALUE=1 + set +o xtrace