e2e-kops-grid-gce-calico-u2004-k26 #1618435509384646656
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 1 failed / 1 succeeded |
| Started | |
| Elapsed | 20m10s |
| Revision | master |
exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest2 Down
... skipping 163 lines ... ........................................Updated [https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4]. ..done. WARNING: No host aliases were added to your SSH configs because you do not have any running instances. Try running this command again after running some instances. I0126 02:29:47.988480 5544 up.go:44] Cleaning up any leaked resources from previous cluster I0126 02:29:47.988632 5544 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/06c7f12a-9d21-11ed-b1bb-1299ca1e1805/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh3627749624/key --ssh-user prow I0126 02:29:47.988648 5544 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/06c7f12a-9d21-11ed-b1bb-1299ca1e1805/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh3627749624/key --ssh-user prow Error: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" not found W0126 02:29:48.287498 5544 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0126 02:29:48.287547 5544 down.go:48] /home/prow/go/src/k8s.io/kops/_rundir/06c7f12a-9d21-11ed-b1bb-1299ca1e1805/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --yes I0126 02:29:48.287560 5544 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/06c7f12a-9d21-11ed-b1bb-1299ca1e1805/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --yes Error: error reading cluster configuration: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" not found I0126 02:29:48.521119 5544 gcs.go:51] gsutil ls -b -p e2e-gce-gci-ci-1-4 gs://e2e-gce-gci-ci-1-4-state-06 I0126 02:29:48.521157 5544 local.go:42] ⚙️ gsutil ls -b -p e2e-gce-gci-ci-1-4 gs://e2e-gce-gci-ci-1-4-state-06 I0126 02:29:50.492954 5544 gcs.go:70] gsutil mb -p e2e-gce-gci-ci-1-4 gs://e2e-gce-gci-ci-1-4-state-06 I0126 02:29:50.492993 5544 local.go:42] ⚙️ gsutil mb -p e2e-gce-gci-ci-1-4 gs://e2e-gce-gci-ci-1-4-state-06 Creating gs://e2e-gce-gci-ci-1-4-state-06/... I0126 02:29:53.148139 5544 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2023/01/26 02:29:53 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I0126 02:29:53.162900 5544 http.go:37] curl https://ip.jsb.workers.dev I0126 02:29:53.252466 5544 up.go:167] /home/prow/go/src/k8s.io/kops/_rundir/06c7f12a-9d21-11ed-b1bb-1299ca1e1805/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.26.1 --ssh-public-key /tmp/kops-ssh3627749624/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 104.154.139.206/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-east4-a --master-size e2-standard-2 --project e2e-gce-gci-ci-1-4 I0126 02:29:53.252508 5544 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/06c7f12a-9d21-11ed-b1bb-1299ca1e1805/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.26.1 --ssh-public-key /tmp/kops-ssh3627749624/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 104.154.139.206/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-east4-a --master-size e2-standard-2 --project e2e-gce-gci-ci-1-4 I0126 02:29:53.305344 5862 create_cluster.go:882] Using SSH public key: /tmp/kops-ssh3627749624/key.pub I0126 02:29:53.544373 5862 new_cluster.go:573] VMs will be configured to use specified Service Account: default I0126 02:29:53.544490 5862 new_cluster.go:1338] Cloud Provider ID: "gce" ... skipping 349 lines ... I0126 02:30:02.676324 5882 keypair.go:226] Issuing new certificate: "etcd-manager-ca-events" I0126 02:30:02.681133 5882 keypair.go:226] Issuing new certificate: "etcd-peers-ca-events" W0126 02:30:02.885893 5882 vfs_keystorereader.go:143] CA private key was not found I0126 02:30:03.043465 5882 keypair.go:226] Issuing new certificate: "kubernetes-ca" I0126 02:30:03.098240 5882 keypair.go:226] Issuing new certificate: "service-account" I0126 02:30:18.326581 5882 executor.go:111] Tasks: 42 done / 67 total; 19 can run W0126 02:30:28.845174 5882 executor.go:139] error running task "ForwardingRule/api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local" (9m49s remaining to succeed): error creating ForwardingRule "api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local": googleapi: Error 400: The resource 'projects/e2e-gce-gci-ci-1-4/regions/us-east4/targetPools/api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local' is not ready, resourceNotReady I0126 02:30:28.845250 5882 executor.go:111] Tasks: 60 done / 67 total; 5 can run I0126 02:30:35.838630 5882 executor.go:111] Tasks: 65 done / 67 total; 2 can run I0126 02:30:48.945799 5882 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0126 02:30:49.010836 5882 update_cluster.go:323] Exporting kubeconfig for cluster kOps has set your kubectl context to e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local ... skipping 7 lines ... * read about installing addons at: https://kops.sigs.k8s.io/addons. I0126 02:30:59.555924 5544 up.go:251] /home/prow/go/src/k8s.io/kops/_rundir/06c7f12a-9d21-11ed-b1bb-1299ca1e1805/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --count 10 --wait 15m0s I0126 02:30:59.555983 5544 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/06c7f12a-9d21-11ed-b1bb-1299ca1e1805/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --count 10 --wait 15m0s Validating cluster e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local W0126 02:31:29.939135 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: i/o timeout W0126 02:31:39.965786 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: connect: connection refused W0126 02:31:49.991799 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: connect: connection refused W0126 02:32:00.018098 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: connect: connection refused W0126 02:32:10.045446 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: connect: connection refused W0126 02:32:20.072544 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: connect: connection refused W0126 02:32:30.099544 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: connect: connection refused W0126 02:32:40.126726 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: connect: connection refused W0126 02:32:50.153380 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: connect: connection refused W0126 02:33:00.179456 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: connect: connection refused W0126 02:33:10.205722 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: connect: connection refused W0126 02:33:30.232543 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": net/http: TLS handshake timeout W0126 02:33:50.259299 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": net/http: TLS handshake timeout W0126 02:34:00.284450 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: connect: connection refused W0126 02:34:10.310703 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: connect: connection refused W0126 02:34:30.337201 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": net/http: TLS handshake timeout I0126 02:34:36.467878 5544 boskos.go:86] Sending heartbeat to Boskos W0126 02:34:40.363324 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: connect: connection refused W0126 02:34:50.389394 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: connect: connection refused W0126 02:35:00.416953 5903 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.150.224.253/api/v1/nodes": dial tcp 34.150.224.253:443: connect: connection refused I0126 02:35:13.074536 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 5 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/control-plane-us-east4-a-0w60 machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/control-plane-us-east4-a-0w60" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-2rbb machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-2rbb" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-h4dq machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-h4dq" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-mz3t machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-mz3t" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-tl4q machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-tl4q" has not yet joined cluster Validation Failed W0126 02:35:13.737200 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:35:24.191658 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 6 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/control-plane-us-east4-a-0w60 machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/control-plane-us-east4-a-0w60" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-2rbb machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-2rbb" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-h4dq machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-h4dq" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-mz3t machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-mz3t" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-tl4q machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-tl4q" has not yet joined cluster Validation Failed W0126 02:35:24.674506 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:35:35.159654 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 10 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-tl4q machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-tl4q" has not yet joined cluster Pod kube-system/calico-kube-controllers-55f488656-ctmmv system-cluster-critical pod "calico-kube-controllers-55f488656-ctmmv" is pending Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is pending Pod kube-system/coredns-7bc44848df-xf72s system-cluster-critical pod "coredns-7bc44848df-xf72s" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-l49g9 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-l49g9" is pending Validation Failed W0126 02:35:35.628061 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:35:46.058017 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 10 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-tl4q machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-tl4q" has not yet joined cluster Pod kube-system/calico-kube-controllers-55f488656-ctmmv system-cluster-critical pod "calico-kube-controllers-55f488656-ctmmv" is pending Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is pending Pod kube-system/coredns-7bc44848df-xf72s system-cluster-critical pod "coredns-7bc44848df-xf72s" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-l49g9 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-l49g9" is pending Validation Failed W0126 02:35:46.631815 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:35:57.081729 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 13 lines ... Pod kube-system/cloud-controller-manager-njnnb system-cluster-critical pod "cloud-controller-manager-njnnb" is pending Pod kube-system/coredns-7bc44848df-xf72s system-cluster-critical pod "coredns-7bc44848df-xf72s" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-l49g9 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-l49g9" is pending Pod kube-system/dns-controller-6d5f4bf5d6-k6fv7 system-cluster-critical pod "dns-controller-6d5f4bf5d6-k6fv7" is pending Pod kube-system/kops-controller-nn2bw system-cluster-critical pod "kops-controller-nn2bw" is pending Validation Failed W0126 02:35:57.556392 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:36:07.979764 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 15 lines ... Pod kube-system/coredns-autoscaler-7cb5c5b969-l49g9 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-l49g9" is pending Pod kube-system/dns-controller-6d5f4bf5d6-k6fv7 system-cluster-critical pod "dns-controller-6d5f4bf5d6-k6fv7" is pending Pod kube-system/etcd-manager-events-control-plane-us-east4-a-0w60 system-cluster-critical pod "etcd-manager-events-control-plane-us-east4-a-0w60" is pending Pod kube-system/kops-controller-nn2bw system-cluster-critical pod "kops-controller-nn2bw" is pending Pod kube-system/kube-scheduler-control-plane-us-east4-a-0w60 system-cluster-critical pod "kube-scheduler-control-plane-us-east4-a-0w60" is pending Validation Failed W0126 02:36:08.490319 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:36:18.964169 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 10 lines ... Machine https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-tl4q machine "https://www.googleapis.com/compute/v1/projects/e2e-gce-gci-ci-1-4/zones/us-east4-a/instances/nodes-us-east4-a-tl4q" has not yet joined cluster Pod kube-system/calico-kube-controllers-55f488656-ctmmv system-cluster-critical pod "calico-kube-controllers-55f488656-ctmmv" is pending Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is pending Pod kube-system/coredns-7bc44848df-xf72s system-cluster-critical pod "coredns-7bc44848df-xf72s" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-l49g9 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-l49g9" is pending Validation Failed W0126 02:36:19.406954 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:36:29.889707 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 11 lines ... Pod kube-system/calico-kube-controllers-55f488656-ctmmv system-cluster-critical pod "calico-kube-controllers-55f488656-ctmmv" is pending Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is pending Pod kube-system/coredns-7bc44848df-xf72s system-cluster-critical pod "coredns-7bc44848df-xf72s" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-l49g9 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-l49g9" is pending Pod kube-system/kube-controller-manager-control-plane-us-east4-a-0w60 system-cluster-critical pod "kube-controller-manager-control-plane-us-east4-a-0w60" is pending Validation Failed W0126 02:36:30.357074 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:36:40.845467 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 15 lines ... Pod kube-system/calico-node-lqn7j system-node-critical pod "calico-node-lqn7j" is pending Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Pod kube-system/coredns-7bc44848df-xf72s system-cluster-critical pod "coredns-7bc44848df-xf72s" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-l49g9 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-l49g9" is pending Pod kube-system/kube-apiserver-control-plane-us-east4-a-0w60 system-cluster-critical pod "kube-apiserver-control-plane-us-east4-a-0w60" is pending Validation Failed W0126 02:36:41.368467 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:36:51.850235 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 18 lines ... Pod kube-system/calico-node-fztzk system-node-critical pod "calico-node-fztzk" is pending Pod kube-system/calico-node-lqn7j system-node-critical pod "calico-node-lqn7j" is pending Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Pod kube-system/coredns-7bc44848df-xf72s system-cluster-critical pod "coredns-7bc44848df-xf72s" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-l49g9 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-l49g9" is pending Validation Failed W0126 02:36:52.429788 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:37:02.915869 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 12 lines ... Pod kube-system/calico-node-c6mrp system-node-critical pod "calico-node-c6mrp" is pending Pod kube-system/calico-node-fztzk system-node-critical pod "calico-node-fztzk" is pending Pod kube-system/calico-node-lqn7j system-node-critical pod "calico-node-lqn7j" is not ready (calico-node) Pod kube-system/coredns-7bc44848df-xf72s system-cluster-critical pod "coredns-7bc44848df-xf72s" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-l49g9 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-l49g9" is pending Validation Failed W0126 02:37:03.536480 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:37:13.949260 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 10 lines ... KIND NAME MESSAGE Pod kube-system/calico-node-7lck5 system-node-critical pod "calico-node-7lck5" is not ready (calico-node) Pod kube-system/calico-node-c6mrp system-node-critical pod "calico-node-c6mrp" is not ready (calico-node) Pod kube-system/calico-node-fztzk system-node-critical pod "calico-node-fztzk" is not ready (calico-node) Pod kube-system/calico-node-lqn7j system-node-critical pod "calico-node-lqn7j" is not ready (calico-node) Validation Failed W0126 02:37:14.432958 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:37:24.903399 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 11 lines ... Pod kube-system/calico-node-7lck5 system-node-critical pod "calico-node-7lck5" is not ready (calico-node) Pod kube-system/calico-node-c6mrp system-node-critical pod "calico-node-c6mrp" is not ready (calico-node) Pod kube-system/calico-node-fztzk system-node-critical pod "calico-node-fztzk" is not ready (calico-node) Pod kube-system/calico-node-lqn7j system-node-critical pod "calico-node-lqn7j" is not ready (calico-node) Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:37:25.381965 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:37:35.758660 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 9 lines ... VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-c6mrp system-node-critical pod "calico-node-c6mrp" is not ready (calico-node) Pod kube-system/calico-node-fztzk system-node-critical pod "calico-node-fztzk" is not ready (calico-node) Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:37:36.184151 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:37:46.616768 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 8 lines ... VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Pod kube-system/kube-proxy-nodes-us-east4-a-mz3t system-node-critical pod "kube-proxy-nodes-us-east4-a-mz3t" is pending Validation Failed W0126 02:37:47.242229 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:37:57.633644 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 8 lines ... VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Pod kube-system/kube-proxy-nodes-us-east4-a-tl4q system-node-critical pod "kube-proxy-nodes-us-east4-a-tl4q" is pending Validation Failed W0126 02:37:58.091023 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:38:08.652052 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:38:09.101173 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:38:19.534921 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:38:20.034387 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:38:30.462210 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:38:31.056470 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:38:41.629855 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:38:42.235405 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:38:52.702005 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:38:53.183424 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:39:03.652330 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:39:04.102327 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:39:14.522485 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:39:14.966581 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:39:25.350695 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:39:25.842552 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:39:36.271391 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] I0126 02:39:36.477527 5544 boskos.go:86] Sending heartbeat to Boskos INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 ... skipping 8 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:39:36.702811 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:39:47.212439 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:39:47.662351 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:39:58.085729 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:39:58.515307 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:40:08.911855 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:40:09.386009 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:40:19.799182 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:40:20.312977 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:40:30.733609 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:40:31.206408 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:40:41.654228 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:40:42.152886 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:40:52.530375 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:40:53.003465 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:41:03.541981 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:41:04.011228 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:41:14.448074 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:41:14.900985 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:41:25.366657 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:41:25.890576 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:41:36.321025 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:41:36.777962 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:41:47.222941 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:41:47.813382 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:41:58.282814 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:41:58.779544 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:42:09.205069 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:42:09.667272 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:42:20.174595 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:42:20.617199 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:42:31.046147 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:42:31.545263 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:42:41.925614 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:42:42.374352 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:42:52.860219 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:42:53.329774 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:43:03.790079 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:43:04.258094 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:43:14.625813 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:43:15.110048 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:43:25.577618 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:43:26.179244 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:43:36.708087 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:43:37.149583 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:43:47.634671 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:43:48.204970 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:43:58.618341 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:43:59.052847 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:44:09.437799 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:44:09.875272 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:44:20.275365 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:44:20.749501 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:44:31.149988 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:44:31.607857 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:44:36.488456 5544 boskos.go:86] Sending heartbeat to Boskos I0126 02:44:42.100148 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 ... skipping 8 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:44:42.681241 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:44:53.074589 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:44:53.580659 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:45:04.028361 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:45:04.438154 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:45:14.844907 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:45:15.282165 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:45:25.683706 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:45:26.176954 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:45:36.645434 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:45:37.143288 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:45:47.535858 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:45:48.015783 5903 validate_cluster.go:232] (will retry): cluster not yet healthy I0126 02:45:58.482879 5903 gce_cloud.go:301] Scanning zones: [us-east4-c us-east4-b us-east4-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-east4-a ControlPlane e2-standard-2 1 1 us-east4 nodes-us-east4-a Node e2-medium 4 4 us-east4 ... skipping 7 lines ... nodes-us-east4-a-tl4q node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rpkh7 system-node-critical pod "calico-node-rpkh7" is not ready (calico-node) Validation Failed W0126 02:45:58.930343 5903 validate_cluster.go:232] (will retry): cluster not yet healthy Error: validation failed: wait time exceeded during validation I0126 02:46:08.949471 5544 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/06c7f12a-9d21-11ed-b1bb-1299ca1e1805/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh3627749624/key --ssh-user prow I0126 02:46:08.949520 5544 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/06c7f12a-9d21-11ed-b1bb-1299ca1e1805/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh3627749624/key --ssh-user prow I0126 02:46:09.436167 5913 gce.go:99] Scanning zones: [us-east4-c us-east4-b us-east4-a] 2023/01/26 02:46:13 Dumping node control-plane-us-east4-a-0w60 2023/01/26 02:46:18 error dumping node control-plane-us-east4-a-0w60: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/01/26 02:46:18 error dumping node control-plane-us-east4-a-0w60: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/01/26 02:46:18 Dumping node nodes-us-east4-a-2rbb 2023/01/26 02:46:21 Dumping node nodes-us-east4-a-h4dq 2023/01/26 02:46:24 Dumping node nodes-us-east4-a-mz3t 2023/01/26 02:46:27 Dumping node nodes-us-east4-a-tl4q I0126 02:46:30.312469 5544 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/06c7f12a-9d21-11ed-b1bb-1299ca1e1805/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local -o yaml I0126 02:46:30.312610 5544 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/06c7f12a-9d21-11ed-b1bb-1299ca1e1805/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local -o yaml ... skipping 154 lines ... Deleted cluster: "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" I0126 02:48:42.319116 5544 gcs.go:88] gsutil -u e2e-gce-gci-ci-1-4 rm -r gs://e2e-gce-gci-ci-1-4-state-06 I0126 02:48:42.319314 5544 local.go:42] ⚙️ gsutil -u e2e-gce-gci-ci-1-4 rm -r gs://e2e-gce-gci-ci-1-4-state-06 Removing gs://e2e-gce-gci-ci-1-4-state-06/... I0126 02:48:44.945927 5544 down.go:62] releasing boskos project I0126 02:48:44.954037 5544 boskos.go:83] Boskos heartbeat func received signal to close Error: exit status 1 + EXIT_VALUE=1 + set +o xtrace