e2e-kops-grid-gce-calico-u2004-k26 #1617589794563428352
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 1 failed / 1 succeeded |
| Started | |
| Elapsed | 19m56s |
| Revision | master |
exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest2 Down
... skipping 163 lines ... .............................Updated [https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n]. ..done. WARNING: No host aliases were added to your SSH configs because you do not have any running instances. Try running this command again after running some instances. I0123 18:29:25.593081 5628 up.go:44] Cleaning up any leaked resources from previous cluster I0123 18:29:25.593245 5628 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/8f79f35b-9b4b-11ed-8662-2ea6bdfefe0d/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh4162991417/key --ssh-user prow I0123 18:29:25.593827 5628 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/8f79f35b-9b4b-11ed-8662-2ea6bdfefe0d/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh4162991417/key --ssh-user prow Error: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" not found W0123 18:29:25.893774 5628 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I0123 18:29:25.893845 5628 down.go:48] /home/prow/go/src/k8s.io/kops/_rundir/8f79f35b-9b4b-11ed-8662-2ea6bdfefe0d/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --yes I0123 18:29:25.893860 5628 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/8f79f35b-9b4b-11ed-8662-2ea6bdfefe0d/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --yes Error: error reading cluster configuration: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" not found I0123 18:29:26.121117 5628 gcs.go:51] gsutil ls -b -p k8s-gce-upg-1-5-1-6-up-clu-n gs://k8s-gce-upg-1-5-1-6-up-clu-n-state-8f I0123 18:29:26.121160 5628 local.go:42] ⚙️ gsutil ls -b -p k8s-gce-upg-1-5-1-6-up-clu-n gs://k8s-gce-upg-1-5-1-6-up-clu-n-state-8f I0123 18:29:28.245049 5628 gcs.go:70] gsutil mb -p k8s-gce-upg-1-5-1-6-up-clu-n gs://k8s-gce-upg-1-5-1-6-up-clu-n-state-8f I0123 18:29:28.245086 5628 local.go:42] ⚙️ gsutil mb -p k8s-gce-upg-1-5-1-6-up-clu-n gs://k8s-gce-upg-1-5-1-6-up-clu-n-state-8f Creating gs://k8s-gce-upg-1-5-1-6-up-clu-n-state-8f/... I0123 18:29:30.975799 5628 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2023/01/23 18:29:31 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I0123 18:29:31.007079 5628 http.go:37] curl https://ip.jsb.workers.dev I0123 18:29:31.107673 5628 up.go:167] /home/prow/go/src/k8s.io/kops/_rundir/8f79f35b-9b4b-11ed-8662-2ea6bdfefe0d/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.26.1 --ssh-public-key /tmp/kops-ssh4162991417/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 35.239.116.242/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-central1-a --master-size e2-standard-2 --project k8s-gce-upg-1-5-1-6-up-clu-n I0123 18:29:31.107705 5628 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/8f79f35b-9b4b-11ed-8662-2ea6bdfefe0d/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.26.1 --ssh-public-key /tmp/kops-ssh4162991417/key.pub --set cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 35.239.116.242/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-central1-a --master-size e2-standard-2 --project k8s-gce-upg-1-5-1-6-up-clu-n I0123 18:29:31.160374 5951 create_cluster.go:882] Using SSH public key: /tmp/kops-ssh4162991417/key.pub I0123 18:29:31.434486 5951 new_cluster.go:573] VMs will be configured to use specified Service Account: default I0123 18:29:31.434600 5951 new_cluster.go:1338] Cloud Provider ID: "gce" ... skipping 349 lines ... I0123 18:29:40.002571 5971 keypair.go:226] Issuing new certificate: "apiserver-aggregator-ca" I0123 18:29:40.002598 5971 keypair.go:226] Issuing new certificate: "etcd-peers-ca-main" W0123 18:29:40.173598 5971 vfs_keystorereader.go:143] CA private key was not found I0123 18:29:40.274894 5971 keypair.go:226] Issuing new certificate: "kubernetes-ca" I0123 18:29:40.379964 5971 keypair.go:226] Issuing new certificate: "service-account" I0123 18:29:52.521014 5971 executor.go:111] Tasks: 42 done / 67 total; 19 can run W0123 18:30:05.475649 5971 executor.go:139] error running task "ForwardingRule/api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local" (9m47s remaining to succeed): error creating ForwardingRule "api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local": googleapi: Error 400: The resource 'projects/k8s-gce-upg-1-5-1-6-up-clu-n/regions/us-central1/targetPools/api-e2e-e2e-kops-grid-gce-calico-u2004-k26-k8s-local' is not ready, resourceNotReady I0123 18:30:05.475703 5971 executor.go:111] Tasks: 60 done / 67 total; 5 can run I0123 18:30:12.371469 5971 executor.go:111] Tasks: 65 done / 67 total; 2 can run I0123 18:30:25.342064 5971 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0123 18:30:25.413370 5971 update_cluster.go:323] Exporting kubeconfig for cluster kOps has set your kubectl context to e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local ... skipping 7 lines ... * read about installing addons at: https://kops.sigs.k8s.io/addons. I0123 18:30:35.777607 5628 up.go:251] /home/prow/go/src/k8s.io/kops/_rundir/8f79f35b-9b4b-11ed-8662-2ea6bdfefe0d/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --count 10 --wait 15m0s I0123 18:30:35.777703 5628 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/8f79f35b-9b4b-11ed-8662-2ea6bdfefe0d/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --count 10 --wait 15m0s Validating cluster e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local W0123 18:31:06.213728 5990 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.27.224.15/api/v1/nodes": dial tcp 34.27.224.15:443: i/o timeout W0123 18:31:46.215602 5990 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.27.224.15/api/v1/nodes": dial tcp 34.27.224.15:443: i/o timeout W0123 18:31:56.220407 5990 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.27.224.15/api/v1/nodes": dial tcp 34.27.224.15:443: connect: connection refused W0123 18:32:06.224341 5990 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.27.224.15/api/v1/nodes": dial tcp 34.27.224.15:443: connect: connection refused W0123 18:32:16.227962 5990 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.27.224.15/api/v1/nodes": dial tcp 34.27.224.15:443: connect: connection refused W0123 18:32:26.231860 5990 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.27.224.15/api/v1/nodes": dial tcp 34.27.224.15:443: connect: connection refused W0123 18:32:46.238517 5990 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.27.224.15/api/v1/nodes": net/http: TLS handshake timeout W0123 18:32:57.289606 5990 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.27.224.15/api/v1/nodes": dial tcp 34.27.224.15:443: connect: connection refused - error from a previous attempt: read tcp 10.60.85.76:55248->34.27.224.15:443: read: connection reset by peer W0123 18:33:17.295250 5990 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.27.224.15/api/v1/nodes": net/http: TLS handshake timeout W0123 18:33:27.300871 5990 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.27.224.15/api/v1/nodes": dial tcp 34.27.224.15:443: connect: connection refused W0123 18:33:37.306026 5990 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.27.224.15/api/v1/nodes": dial tcp 34.27.224.15:443: connect: connection refused W0123 18:33:57.312926 5990 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.27.224.15/api/v1/nodes": net/http: TLS handshake timeout W0123 18:34:07.317880 5990 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.27.224.15/api/v1/nodes": dial tcp 34.27.224.15:443: connect: connection refused I0123 18:34:15.970757 5628 boskos.go:86] Sending heartbeat to Boskos W0123 18:34:17.324113 5990 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.27.224.15/api/v1/nodes": dial tcp 34.27.224.15:443: connect: connection refused I0123 18:34:27.738866 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 5 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/control-plane-us-central1-a-65vx machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/control-plane-us-central1-a-65vx" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-3wjp machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-3wjp" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-7cjk machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-7cjk" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-gbhw machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-gbhw" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-nw6z machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-nw6z" has not yet joined cluster Validation Failed W0123 18:34:28.382006 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:34:38.824698 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 6 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/control-plane-us-central1-a-65vx machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/control-plane-us-central1-a-65vx" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-3wjp machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-3wjp" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-7cjk machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-7cjk" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-gbhw machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-gbhw" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-nw6z machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-nw6z" has not yet joined cluster Validation Failed W0123 18:34:39.374585 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:34:49.868854 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 6 lines ... Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/control-plane-us-central1-a-65vx machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/control-plane-us-central1-a-65vx" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-3wjp machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-3wjp" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-7cjk machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-7cjk" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-gbhw machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-gbhw" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-nw6z machine "https://www.googleapis.com/compute/v1/projects/k8s-gce-upg-1-5-1-6-up-clu-n/zones/us-central1-a/instances/nodes-us-central1-a-nw6z" has not yet joined cluster Validation Failed W0123 18:34:50.476440 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:35:00.941796 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 11 lines ... Pod kube-system/calico-kube-controllers-55f488656-2fdpd system-cluster-critical pod "calico-kube-controllers-55f488656-2fdpd" is pending Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is pending Pod kube-system/coredns-7bc44848df-2gbn5 system-cluster-critical pod "coredns-7bc44848df-2gbn5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-hmw76 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-hmw76" is pending Pod kube-system/dns-controller-6d5f4bf5d6-kbqtx system-cluster-critical pod "dns-controller-6d5f4bf5d6-kbqtx" is pending Validation Failed W0123 18:35:01.487489 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:35:11.896099 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 13 lines ... Pod kube-system/cloud-controller-manager-b2bgw system-cluster-critical pod "cloud-controller-manager-b2bgw" is pending Pod kube-system/coredns-7bc44848df-2gbn5 system-cluster-critical pod "coredns-7bc44848df-2gbn5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-hmw76 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-hmw76" is pending Pod kube-system/dns-controller-6d5f4bf5d6-kbqtx system-cluster-critical pod "dns-controller-6d5f4bf5d6-kbqtx" is pending Pod kube-system/kops-controller-jmdgf system-cluster-critical pod "kops-controller-jmdgf" is pending Validation Failed W0123 18:35:12.460262 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:35:22.856340 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 11 lines ... Pod kube-system/calico-kube-controllers-55f488656-2fdpd system-cluster-critical pod "calico-kube-controllers-55f488656-2fdpd" is pending Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is pending Pod kube-system/cloud-controller-manager-b2bgw system-cluster-critical pod "cloud-controller-manager-b2bgw" is pending Pod kube-system/coredns-7bc44848df-2gbn5 system-cluster-critical pod "coredns-7bc44848df-2gbn5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-hmw76 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-hmw76" is pending Validation Failed W0123 18:35:23.488998 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:35:33.913242 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 11 lines ... Pod kube-system/calico-kube-controllers-55f488656-2fdpd system-cluster-critical pod "calico-kube-controllers-55f488656-2fdpd" is pending Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is pending Pod kube-system/coredns-7bc44848df-2gbn5 system-cluster-critical pod "coredns-7bc44848df-2gbn5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-hmw76 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-hmw76" is pending Pod kube-system/kube-apiserver-control-plane-us-central1-a-65vx system-cluster-critical pod "kube-apiserver-control-plane-us-central1-a-65vx" is pending Validation Failed W0123 18:35:34.508092 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:35:44.886519 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 19 lines ... Pod kube-system/calico-node-mf678 system-node-critical pod "calico-node-mf678" is pending Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is pending Pod kube-system/calico-node-vslfg system-node-critical pod "calico-node-vslfg" is pending Pod kube-system/coredns-7bc44848df-2gbn5 system-cluster-critical pod "coredns-7bc44848df-2gbn5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-hmw76 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-hmw76" is pending Validation Failed W0123 18:35:45.390543 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:35:55.804713 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 16 lines ... Pod kube-system/calico-node-lf9bj system-node-critical pod "calico-node-lf9bj" is pending Pod kube-system/calico-node-mf678 system-node-critical pod "calico-node-mf678" is pending Pod kube-system/calico-node-vslfg system-node-critical pod "calico-node-vslfg" is pending Pod kube-system/coredns-7bc44848df-2gbn5 system-cluster-critical pod "coredns-7bc44848df-2gbn5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-hmw76 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-hmw76" is pending Validation Failed W0123 18:35:56.303062 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:36:06.682517 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 15 lines ... Pod kube-system/calico-node-vslfg system-node-critical pod "calico-node-vslfg" is not ready (calico-node) Pod kube-system/coredns-7bc44848df-2gbn5 system-cluster-critical pod "coredns-7bc44848df-2gbn5" is pending Pod kube-system/coredns-autoscaler-7cb5c5b969-hmw76 system-cluster-critical pod "coredns-autoscaler-7cb5c5b969-hmw76" is pending Pod kube-system/kube-proxy-control-plane-us-central1-a-65vx system-node-critical pod "kube-proxy-control-plane-us-central1-a-65vx" is pending Pod kube-system/kube-scheduler-control-plane-us-central1-a-65vx system-cluster-critical pod "kube-scheduler-control-plane-us-central1-a-65vx" is pending Validation Failed W0123 18:36:07.267700 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:36:17.703243 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 11 lines ... Pod kube-system/calico-node-j6pvx system-node-critical pod "calico-node-j6pvx" is not ready (calico-node) Pod kube-system/calico-node-lf9bj system-node-critical pod "calico-node-lf9bj" is not ready (calico-node) Pod kube-system/calico-node-mf678 system-node-critical pod "calico-node-mf678" is not ready (calico-node) Pod kube-system/calico-node-vslfg system-node-critical pod "calico-node-vslfg" is not ready (calico-node) Pod kube-system/etcd-manager-events-control-plane-us-central1-a-65vx system-cluster-critical pod "etcd-manager-events-control-plane-us-central1-a-65vx" is pending Validation Failed W0123 18:36:18.211576 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:36:28.633668 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 11 lines ... Pod kube-system/calico-node-j6pvx system-node-critical pod "calico-node-j6pvx" is not ready (calico-node) Pod kube-system/calico-node-lf9bj system-node-critical pod "calico-node-lf9bj" is not ready (calico-node) Pod kube-system/calico-node-mf678 system-node-critical pod "calico-node-mf678" is not ready (calico-node) Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Pod kube-system/calico-node-vslfg system-node-critical pod "calico-node-vslfg" is not ready (calico-node) Validation Failed W0123 18:36:29.204595 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:36:39.550790 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 10 lines ... KIND NAME MESSAGE Pod kube-system/calico-node-j6pvx system-node-critical pod "calico-node-j6pvx" is not ready (calico-node) Pod kube-system/calico-node-lf9bj system-node-critical pod "calico-node-lf9bj" is not ready (calico-node) Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Pod kube-system/calico-node-vslfg system-node-critical pod "calico-node-vslfg" is not ready (calico-node) Validation Failed W0123 18:36:40.099986 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:36:50.468584 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 9 lines ... VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Pod kube-system/kube-proxy-nodes-us-central1-a-3wjp system-node-critical pod "kube-proxy-nodes-us-central1-a-3wjp" is pending Pod kube-system/kube-proxy-nodes-us-central1-a-gbhw system-node-critical pod "kube-proxy-nodes-us-central1-a-gbhw" is pending Validation Failed W0123 18:36:50.981827 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:37:01.367443 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:37:01.936604 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:37:12.350385 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:37:12.938084 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:37:23.428327 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:37:23.936265 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:37:34.341666 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:37:34.897572 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:37:45.285743 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:37:45.781723 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:37:56.130913 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:37:56.690301 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:38:07.105072 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:38:07.639212 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:38:18.003229 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:38:18.562220 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:38:28.946399 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:38:29.466895 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:38:39.887598 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:38:40.499870 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:38:50.930841 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:38:51.450387 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:39:01.825382 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:39:02.347604 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:39:12.798254 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:39:13.322727 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:39:15.981695 5628 boskos.go:86] Sending heartbeat to Boskos I0123 18:39:23.799024 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 8 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:39:24.359500 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:39:34.833368 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:39:35.395970 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:39:45.860495 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:39:46.403180 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:39:56.777312 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:39:57.375696 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:40:07.768033 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:40:08.289798 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:40:18.673671 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:40:19.249959 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:40:29.644244 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:40:30.203011 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:40:40.657855 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:40:41.265103 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:40:51.630594 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:40:52.222505 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:41:02.587387 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:41:03.097577 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:41:13.497752 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:41:14.051810 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:41:24.427702 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:41:24.937089 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:41:35.337076 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:41:35.894864 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:41:46.367920 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:41:46.930914 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:41:57.304240 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:41:57.853955 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:42:08.374615 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:42:08.897244 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:42:19.366103 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:42:19.884389 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:42:30.304352 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:42:30.822852 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:42:41.241233 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:42:41.782739 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:42:52.157691 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:42:52.769335 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:43:03.137284 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:43:03.714423 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:43:14.113850 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:43:14.674685 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:43:25.031852 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:43:25.681132 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:43:36.042587 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:43:36.598693 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:43:46.954519 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:43:47.467688 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:43:57.900653 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:43:58.484131 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:44:08.892756 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:44:09.433037 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:44:15.993181 5628 boskos.go:86] Sending heartbeat to Boskos I0123 18:44:19.837284 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 ... skipping 8 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:44:20.415582 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:44:30.792333 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:44:31.336009 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:44:41.750054 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:44:42.304354 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:44:52.751085 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:44:53.308512 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:45:03.787974 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:45:04.432217 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:45:14.913113 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:45:15.458168 5990 validate_cluster.go:232] (will retry): cluster not yet healthy I0123 18:45:25.902165 5990 gce_cloud.go:301] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS control-plane-us-central1-a ControlPlane e2-standard-2 1 1 us-central1 nodes-us-central1-a Node e2-medium 4 4 us-central1 ... skipping 7 lines ... nodes-us-central1-a-nw6z node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-rkqct system-node-critical pod "calico-node-rkqct" is not ready (calico-node) Validation Failed W0123 18:45:26.442158 5990 validate_cluster.go:232] (will retry): cluster not yet healthy Error: validation failed: wait time exceeded during validation I0123 18:45:36.451476 5628 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/8f79f35b-9b4b-11ed-8662-2ea6bdfefe0d/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh4162991417/key --ssh-user prow I0123 18:45:36.451539 5628 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/8f79f35b-9b4b-11ed-8662-2ea6bdfefe0d/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh4162991417/key --ssh-user prow I0123 18:45:36.913123 6000 gce.go:99] Scanning zones: [us-central1-c us-central1-a us-central1-f us-central1-b us-central1-d] 2023/01/23 18:45:40 Dumping node control-plane-us-central1-a-65vx 2023/01/23 18:45:44 error dumping node control-plane-us-central1-a-65vx: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=external-dns\"; fi": Process exited with status 1 2023/01/23 18:45:44 error dumping node control-plane-us-central1-a-65vx: error executing command "if command -v kubectl &> /dev/null; then kubectl logs -n kube-system --all-containers -l \"k8s-app=dns-controller\"; fi": Process exited with status 1 2023/01/23 18:45:44 Dumping node nodes-us-central1-a-3wjp 2023/01/23 18:45:45 Dumping node nodes-us-central1-a-7cjk 2023/01/23 18:45:48 Dumping node nodes-us-central1-a-gbhw 2023/01/23 18:45:50 Dumping node nodes-us-central1-a-nw6z I0123 18:45:52.765330 5628 dumplogs.go:79] /home/prow/go/src/k8s.io/kops/_rundir/8f79f35b-9b4b-11ed-8662-2ea6bdfefe0d/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local -o yaml I0123 18:45:52.765382 5628 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/8f79f35b-9b4b-11ed-8662-2ea6bdfefe0d/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local -o yaml ... skipping 155 lines ... Deleted cluster: "e2e-e2e-kops-grid-gce-calico-u2004-k26.k8s.local" I0123 18:48:01.195932 5628 gcs.go:88] gsutil -u k8s-gce-upg-1-5-1-6-up-clu-n rm -r gs://k8s-gce-upg-1-5-1-6-up-clu-n-state-8f I0123 18:48:01.196006 5628 local.go:42] ⚙️ gsutil -u k8s-gce-upg-1-5-1-6-up-clu-n rm -r gs://k8s-gce-upg-1-5-1-6-up-clu-n-state-8f Removing gs://k8s-gce-upg-1-5-1-6-up-clu-n-state-8f/... I0123 18:48:03.912742 5628 down.go:62] releasing boskos project I0123 18:48:03.921490 5628 boskos.go:83] Boskos heartbeat func received signal to close Error: exit status 1 + EXIT_VALUE=1 + set +o xtrace