e2e-kops-grid-gce-calico-u2004-k26 #1614811578916409344
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 1 failed / 1 succeeded |
| Started | |
| Elapsed | 20m32s |
| Revision | master |
exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest2 Down
... skipping 163 lines ... ..............................Updated [https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew]. ..done. WARNING: No host aliases were added to your SSH configs because you do not have any running instances. Try running this command again after running some instances. I0116 02:29:48.955106 5513 up.go:44] Cleaning up any leaked resources from previous cluster I0116 02:29:48.955298 5513 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/57495459-9545-11ed-bd5d-72eebb4d772a/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh3422147351/key --ssh-user prow I0116 02:29:48.955374 5513 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/57495459-9545-11ed-bd5d-72eebb4d772a/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh3422147351/key --ssh-user prow Error: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" not found W0116 02:29:49.270495 5513 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0116 02:29:49.270568 5513 down.go:48] /home/prow/go/src/k8s.io/kops/_rundir/57495459-9545-11ed-bd5d-72eebb4d772a/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --yes I0116 02:29:49.270581 5513 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/57495459-9545-11ed-bd5d-72eebb4d772a/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --yes Error: error reading cluster configuration: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" not found I0116 02:29:49.542420 5513 gcs.go:51] gsutil ls -b -p k8s-gce-gci-1-6-mstr-ctl-skew gs://k8s-gce-gci-1-6-mstr-ctl-skew-state-57 I0116 02:29:49.542482 5513 local.go:42] ⚙️ gsutil ls -b -p k8s-gce-gci-1-6-mstr-ctl-skew gs://k8s-gce-gci-1-6-mstr-ctl-skew-state-57 I0116 02:29:52.356227 5513 gcs.go:70] gsutil mb -p k8s-gce-gci-1-6-mstr-ctl-skew gs://k8s-gce-gci-1-6-mstr-ctl-skew-state-57 I0116 02:29:52.356294 5513 local.go:42] ⚙️ gsutil mb -p k8s-gce-gci-1-6-mstr-ctl-skew gs://k8s-gce-gci-1-6-mstr-ctl-skew-state-57 Creating gs://k8s-gce-gci-1-6-mstr-ctl-skew-state-57/... I0116 02:29:55.215511 5513 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2023/01/16 02:29:55 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I0116 02:29:55.229458 5513 http.go:37] curl https://ip.jsb.workers.dev I0116 02:29:55.379227 5513 up.go:167] /home/prow/go/src/k8s.io/kops/_rundir/57495459-9545-11ed-bd5d-72eebb4d772a/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.26.0 --ssh-public-key /tmp/kops-ssh3422147351/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 34.136.112.132/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-west4-a --master-size e2-standard-2 --project k8s-gce-gci-1-6-mstr-ctl-skew I0116 02:29:55.379275 5513 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/57495459-9545-11ed-bd5d-72eebb4d772a/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.26.0 --ssh-public-key /tmp/kops-ssh3422147351/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 34.136.112.132/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-west4-a --master-size e2-standard-2 --project k8s-gce-gci-1-6-mstr-ctl-skew I0116 02:29:55.440357 5839 create_cluster.go:882] Using SSH public key: /tmp/kops-ssh3422147351/key.pub I0116 02:29:55.825069 5839 new_cluster.go:573] VMs will be configured to use specified Service Account: default I0116 02:29:55.825263 5839 new_cluster.go:1338] Cloud Provider ID: "gce" ... skipping 349 lines ... I0116 02:30:04.188496 5859 keypair.go:226] Issuing new certificate: "etcd-peers-ca-events" I0116 02:30:04.202388 5859 address.go:139] GCE creating address: "api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local" W0116 02:30:04.276626 5859 vfs_keystorereader.go:143] CA private key was not found I0116 02:30:04.370392 5859 keypair.go:226] Issuing new certificate: "kubernetes-ca" I0116 02:30:04.470525 5859 keypair.go:226] Issuing new certificate: "service-account" I0116 02:30:19.821319 5859 executor.go:111] Tasks: 42 done / 67 total; 19 can run W0116 02:30:35.115787 5859 executor.go:139] error running task "ForwardingRule/api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local" (9m44s remaining to succeed): error creating ForwardingRule "api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local": googleapi: Error 400: The resource 'projects/k8s-gce-gci-1-6-mstr-ctl-skew/regions/us-west4/targetPools/api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local' is not ready, resourceNotReady I0116 02:30:35.115846 5859 executor.go:111] Tasks: 60 done / 67 total; 5 can run I0116 02:30:43.047772 5859 executor.go:111] Tasks: 65 done / 67 total; 2 can run I0116 02:30:59.247128 5859 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0116 02:30:59.317421 5859 update_cluster.go:323] Exporting kubeconfig for cluster kOps has set your kubectl context to e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local ... skipping 7 lines ... * read about installing addons at: https://kops.sigs.k8s.io/addons. I0116 02:31:10.070269 5513 up.go:251] /home/prow/go/src/k8s.io/kops/_rundir/57495459-9545-11ed-bd5d-72eebb4d772a/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --count 10 --wait 15m0s I0116 02:31:10.070335 5513 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/57495459-9545-11ed-bd5d-72eebb4d772a/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --count 10 --wait 15m0s Validating cluster e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local W0116 02:31:40.424102 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": dial tcp 34.125.167.160:443: i/o timeout W0116 02:31:50.468236 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": dial tcp 34.125.167.160:443: connect: connection refused W0116 02:32:00.511193 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": dial tcp 34.125.167.160:443: connect: connection refused W0116 02:32:10.557009 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": dial tcp 34.125.167.160:443: connect: connection refused W0116 02:32:20.601414 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": dial tcp 34.125.167.160:443: connect: connection refused W0116 02:32:30.645051 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": dial tcp 34.125.167.160:443: connect: connection refused W0116 02:32:40.688779 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": dial tcp 34.125.167.160:443: connect: connection refused W0116 02:32:50.731439 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": dial tcp 34.125.167.160:443: connect: connection refused W0116 02:33:00.774459 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": dial tcp 34.125.167.160:443: connect: connection refused W0116 02:33:10.817906 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": dial tcp 34.125.167.160:443: connect: connection refused W0116 02:33:20.860856 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": dial tcp 34.125.167.160:443: connect: connection refused W0116 02:33:30.904291 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": dial tcp 34.125.167.160:443: connect: connection refused W0116 02:33:50.948708 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": net/http: TLS handshake timeout W0116 02:34:10.992688 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": net/http: TLS handshake timeout W0116 02:34:21.035047 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": dial tcp 34.125.167.160:443: connect: connection refused W0116 02:34:31.077609 5879 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.125.167.160/api/v1/nodes": dial tcp 34.125.167.160:443: connect: connection refused I0116 02:34:39.055270 5513 boskos.go:86] Sending heartbeat to Boskos I0116 02:34:41.713736 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 6 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/control-plane-us-west4-a-zd58 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/control-plane-us-west4-a-zd58" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-6c5f machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-6c5f" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-j9q3 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-j9q3" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-rmf6 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-rmf6" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-zvns machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-zvns" has not yet joined cluster Validation Failed W0116 02:34:42.697248 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:34:53.008696 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 6 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/control-plane-us-west4-a-zd58 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/control-plane-us-west4-a-zd58" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-6c5f machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-6c5f" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-j9q3 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-j9q3" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-rmf6 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-rmf6" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-zvns machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-zvns" has not yet joined cluster Validation Failed W0116 02:34:53.715094 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:35:04.149887 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-j9q3 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-j9q3" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-rmf6 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-rmf6" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-zvns machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-zvns" has not yet joined cluster Pod kube-system/coredns-7bc44848df-n5dw5 system-cluster-critical pod "coredns-7bc44848df-n5dw5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-zd2rk system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-zd2rk" is pending Validation Failed W0116 02:35:04.985762 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:35:15.339876 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 8 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-j9q3 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-j9q3" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-rmf6 machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-rmf6" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-zvns machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-zvns" has not yet joined cluster Pod kube-system/coredns-7bc44848df-n5dw5 system-cluster-critical pod "coredns-7bc44848df-n5dw5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-zd2rk system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-zd2rk" is pending Validation Failed W0116 02:35:16.538576 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:35:26.871675 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 13 lines ... Pod kube-system/cloud-controller-manager-m6578 system-cluster-critical pod "cloud-controller-manager-m6578" is pending Pod kube-system/coredns-7bc44848df-n5dw5 system-cluster-critical pod "coredns-7bc44848df-n5dw5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-zd2rk system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-zd2rk" is pending Pod kube-system/dns-controller-6d5f4bf5d6-djv8q system-cluster-critical pod "dns-controller-6d5f4bf5d6-djv8q" is pending Pod kube-system/kops-controller-tm2pq system-cluster-critical pod "kops-controller-tm2pq" is pending Validation Failed W0116 02:35:27.548294 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:35:37.893537 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 11 lines ... Pod kube-system/calico-kube-controllers-55f488656-7vhd7 system-cluster-critical pod "calico-kube-controllers-55f488656-7vhd7" is pending Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is pending Pod kube-system/cloud-controller-manager-m6578 system-cluster-critical pod "cloud-controller-manager-m6578" is pending Pod kube-system/coredns-7bc44848df-n5dw5 system-cluster-critical pod "coredns-7bc44848df-n5dw5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-zd2rk system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-zd2rk" is pending Validation Failed W0116 02:35:38.783982 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:35:49.117331 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 10 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-zvns machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-gci-1-6-mstr-ctl-skew/zones/us-west4-a/instances/nodes-us-west4-a-zvns" has not yet joined cluster Pod kube-system/calico-kube-controllers-55f488656-7vhd7 system-cluster-critical pod "calico-kube-controllers-55f488656-7vhd7" is pending Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is pending Pod kube-system/coredns-7bc44848df-n5dw5 system-cluster-critical pod "coredns-7bc44848df-n5dw5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-zd2rk system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-zd2rk" is pending Validation Failed W0116 02:35:49.914887 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:36:00.383470 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 11 lines ... Node control-plane-us-west4-a-zd58 node "control-plane-us-west4-a-zd58" of role "control-plane" is not ready Pod kube-system/calico-kube-controllers-55f488656-7vhd7 system-cluster-critical pod "calico-kube-controllers-55f488656-7vhd7" is pending Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is pending Pod kube-system/coredns-7bc44848df-n5dw5 system-cluster-critical pod "coredns-7bc44848df-n5dw5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-zd2rk system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-zd2rk" is pending Validation Failed W0116 02:36:01.026565 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:36:11.340595 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 15 lines ... Pod kube-system/calico-node-p7vs2 system-node-critical pod "calico-node-p7vs2" is pending Pod kube-system/calico-node-r9rrk system-node-critical pod "calico-node-r9rrk" is pending Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Pod kube-system/coredns-7bc44848df-n5dw5 system-cluster-critical pod "coredns-7bc44848df-n5dw5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-zd2rk system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-zd2rk" is pending Validation Failed W0116 02:36:11.931452 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:36:22.268586 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 16 lines ... Pod kube-system/calico-node-p7vs2 system-node-critical pod "calico-node-p7vs2" is pending Pod kube-system/calico-node-r9rrk system-node-critical pod "calico-node-r9rrk" is pending Pod kube-system/calico-node-wzhsn system-node-critical pod "calico-node-wzhsn" is pending Pod kube-system/coredns-7bc44848df-n5dw5 system-cluster-critical pod "coredns-7bc44848df-n5dw5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-zd2rk system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-zd2rk" is pending Validation Failed W0116 02:36:23.191578 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:36:33.572308 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 16 lines ... Pod kube-system/calico-node-p7vs2 system-node-critical pod "calico-node-p7vs2" is pending Pod kube-system/calico-node-r9rrk system-node-critical pod "calico-node-r9rrk" is pending Pod kube-system/calico-node-wzhsn system-node-critical pod "calico-node-wzhsn" is pending Pod kube-system/coredns-7bc44848df-n5dw5 system-cluster-critical pod "coredns-7bc44848df-n5dw5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-zd2rk system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-zd2rk" is pending Validation Failed W0116 02:36:34.398720 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:36:44.724953 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 13 lines ... Pod kube-system/calico-node-p7vs2 system-node-critical pod "calico-node-p7vs2" is pending Pod kube-system/calico-node-r9rrk system-node-critical pod "calico-node-r9rrk" is pending Pod kube-system/calico-node-wzhsn system-node-critical pod "calico-node-wzhsn" is pending Pod kube-system/coredns-7bc44848df-n5dw5 system-cluster-critical pod "coredns-7bc44848df-n5dw5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-zd2rk system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-zd2rk" is pending Validation Failed W0116 02:36:45.341858 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:36:55.841598 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 12 lines ... Pod kube-system/calico-node-bkpfc system-node-critical pod "calico-node-bkpfc" is pending Pod kube-system/calico-node-p7vs2 system-node-critical pod "calico-node-p7vs2" is pending Pod kube-system/calico-node-r9rrk system-node-critical pod "calico-node-r9rrk" is not ready (calico-node) Pod kube-system/calico-node-wzhsn system-node-critical pod "calico-node-wzhsn" is not ready (calico-node) Pod kube-system/coredns-7bc44848df-fx5qs system-cluster-critical pod "coredns-7bc44848df-fx5qs" is pending Validation Failed W0116 02:36:56.479413 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:37:06.882614 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 10 lines ... KIND NAME MESSAGE Pod kube-system/calico-node-bkpfc system-node-critical pod "calico-node-bkpfc" is not ready (calico-node) Pod kube-system/calico-node-p7vs2 system-node-critical pod "calico-node-p7vs2" is pending Pod kube-system/calico-node-r9rrk system-node-critical pod "calico-node-r9rrk" is not ready (calico-node) Pod kube-system/calico-node-wzhsn system-node-critical pod "calico-node-wzhsn" is not ready (calico-node) Validation Failed W0116 02:37:07.772439 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:37:18.204273 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 11 lines ... Pod kube-system/calico-node-bkpfc system-node-critical pod "calico-node-bkpfc" is not ready (calico-node) Pod kube-system/calico-node-p7vs2 system-node-critical pod "calico-node-p7vs2" is not ready (calico-node) Pod kube-system/calico-node-r9rrk system-node-critical pod "calico-node-r9rrk" is not ready (calico-node) Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Pod kube-system/calico-node-wzhsn system-node-critical pod "calico-node-wzhsn" is not ready (calico-node) Validation Failed W0116 02:37:19.246715 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:37:29.601502 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 10 lines ... KIND NAME MESSAGE Pod kube-system/calico-node-bkpfc system-node-critical pod "calico-node-bkpfc" is not ready (calico-node) Pod kube-system/calico-node-p7vs2 system-node-critical pod "calico-node-p7vs2" is not ready (calico-node) Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Pod kube-system/kube-proxy-nodes-us-west4-a-zvns system-node-critical pod "kube-proxy-nodes-us-west4-a-zvns" is pending Validation Failed W0116 02:37:30.328883 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:37:40.777873 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:37:41.327062 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:37:51.652812 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:37:52.407341 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:38:02.837336 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:38:03.483658 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:38:13.901667 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:38:14.507458 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:38:24.822294 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:38:25.707295 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:38:36.051418 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:38:36.620980 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:38:47.046019 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:38:47.826934 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:38:58.148649 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:38:58.730698 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:39:09.066113 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:39:09.672266 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:39:20.030263 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:39:20.638273 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:39:31.039306 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:39:31.657882 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:39:39.070466 5513 boskos.go:86] Sending heartbeat to Boskos I0116 02:39:42.055478 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 ... skipping 8 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:39:42.658983 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:39:53.110896 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:39:53.946950 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:40:04.258271 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:40:05.075511 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:40:15.489480 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:40:16.054356 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:40:26.470949 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:40:27.043210 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:40:37.511317 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:40:38.160969 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:40:48.514531 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:40:49.057567 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:40:59.403345 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:40:59.977470 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:41:10.352848 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:41:10.940562 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:41:21.291526 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:41:21.931336 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:41:32.274412 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:41:33.119425 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:41:43.459376 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:41:44.102729 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:41:54.496221 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:41:55.132540 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:42:05.470131 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:42:06.112437 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:42:16.482979 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:42:17.125776 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:42:27.586622 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:42:28.179986 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:42:38.508133 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:42:39.053001 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:42:49.513917 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:42:50.437072 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:43:00.758673 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:43:01.279805 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:43:11.609869 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:43:12.205156 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:43:22.589808 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:43:23.235968 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:43:33.629098 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:43:34.429077 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:43:44.780479 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:43:45.375116 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:43:55.866449 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:43:56.678495 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:44:07.059871 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:44:07.849046 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:44:18.164969 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:44:18.834424 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:44:29.264518 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:44:29.912766 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:44:39.081272 5513 boskos.go:86] Sending heartbeat to Boskos I0116 02:44:40.344608 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 ... skipping 8 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:44:40.948731 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:44:51.283845 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:44:51.859380 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:45:02.195367 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:45:02.810849 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:45:13.139027 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:45:13.696952 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:45:24.047295 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:45:24.673972 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:45:35.012226 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:45:35.727619 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:45:46.162414 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:45:46.705917 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:45:57.051443 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:45:57.708632 5879 validate_cluster.go:232] (will retry): cluster not yet healthy I0116 02:46:08.048100 5879 gce_cloud.go:301] Scanning zones: [us-west4-a us-west4-b us-west4-c] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-west4-a ControlPlane e2-standard-2 1 1 us-west4 nodes-us-west4-a Node e2-medium 4 4 us-west4 ... skipping 7 lines ... nodes-us-west4-a-zvns node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-vsfh5 system-node-critical pod "calico-node-vsfh5" is not ready (calico-node) Validation Failed W0116 02:46:08.716520 5879 validate_cluster.go:232] (will retry): cluster not yet healthy Error: validation failed: wait time exceeded during validation I0116 02:46:18.728812 5513 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/57495459-9545-11ed-bd5d-72eebb4d772a/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh3422147351/key --ssh-user prow I0116 02:46:18.728864 5513 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/57495459-9545-11ed-bd5d-72eebb4d772a/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh3422147351/key --ssh-user prow I0116 02:46:19.147224 5889 gce.go:99] Scanning zones: [us-west4-a us-west4-b us-west4-c] 2023/01/16 02:46:24 Dumping node control-plane-us-west4-a-zd58 2023/01/16 02:46:29 error dumping node control-plane-us-west4-a-zd58: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/01/16 02:46:29 error dumping node control-plane-us-west4-a-zd58: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/01/16 02:46:29 Dumping node nodes-us-west4-a-6c5f 2023/01/16 02:46:33 Dumping node nodes-us-west4-a-j9q3 2023/01/16 02:46:37 Dumping node nodes-us-west4-a-rmf6 2023/01/16 02:46:40 Dumping node nodes-us-west4-a-zvns I0116 02:46:44.299929 5513 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/57495459-9545-11ed-bd5d-72eebb4d772a/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local -o yaml I0116 02:46:44.299979 5513 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/57495459-9545-11ed-bd5d-72eebb4d772a/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local -o yaml ... skipping 154 lines ... Deleted cluster: "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" I0116 02:48:59.084339 5513 gcs.go:88] gsutil -u k8s-gce-gci-1-6-mstr-ctl-skew rm -r gs://k8s-gce-gci-1-6-mstr-ctl-skew-state-57 I0116 02:48:59.084405 5513 local.go:42] ⚙️ gsutil -u k8s-gce-gci-1-6-mstr-ctl-skew rm -r gs://k8s-gce-gci-1-6-mstr-ctl-skew-state-57 Removing gs://k8s-gce-gci-1-6-mstr-ctl-skew-state-57/... I0116 02:49:01.897219 5513 down.go:62] releasing boskos project I0116 02:49:01.906485 5513 boskos.go:83] Boskos heartbeat func received signal to close Error: exit status 1 + EXIT_VALUE=1 + set +o xtrace