e2e-kops-grid-gce-calico-u2004-k23-containerd #1589268971691970560
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 1 failed / 1 succeeded |
| Started | |
| Elapsed | 19m38s |
| Revision | master |
kubetest2 Down
... skipping 195 lines ... ..................................Updated [https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew]. .done. WARNING: No host aliases were added to your SSH configs because you do not have any running instances. Try running this command again after running some instances. I1106 14:52:17.345500 6127 up.go:44] Cleaning up any leaked resources from previous cluster I1106 14:52:17.345627 6127 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/587bf420-5de2-11ed-8af1-9a8d20cfcfb2/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1361984811/key --ssh-user prow I1106 14:52:17.345653 6127 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/587bf420-5de2-11ed-8af1-9a8d20cfcfb2/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1361984811/key --ssh-user prow W1106 14:52:17.682046 6127 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1 I1106 14:52:17.682115 6127 down.go:48] /home/prow/go/src/k8s.io/kops/_rundir/587bf420-5de2-11ed-8af1-9a8d20cfcfb2/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local --yes I1106 14:52:17.682130 6127 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/587bf420-5de2-11ed-8af1-9a8d20cfcfb2/kops delete cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local --yes I1106 14:52:17.712623 6171 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true Error: error reading cluster configuration: Cluster.kops.k8s.io "e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local" not found I1106 14:52:17.937856 6127 gcs.go:51] gsutil ls -b -p gce-gci-upg-1-4-1-3-ctl-skew gs://gce-gci-upg-1-4-1-3-ctl-skew-state-58 I1106 14:52:17.937892 6127 local.go:42] ⚙️ gsutil ls -b -p gce-gci-upg-1-4-1-3-ctl-skew gs://gce-gci-upg-1-4-1-3-ctl-skew-state-58 I1106 14:52:20.521454 6127 gcs.go:70] gsutil mb -p gce-gci-upg-1-4-1-3-ctl-skew gs://gce-gci-upg-1-4-1-3-ctl-skew-state-58 I1106 14:52:20.521492 6127 local.go:42] ⚙️ gsutil mb -p gce-gci-upg-1-4-1-3-ctl-skew gs://gce-gci-upg-1-4-1-3-ctl-skew-state-58 Creating gs://gce-gci-upg-1-4-1-3-ctl-skew-state-58/... I1106 14:52:22.903772 6127 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip 2022/11/06 14:52:22 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404 I1106 14:52:22.915832 6127 http.go:37] curl https://ip.jsb.workers.dev I1106 14:52:23.035507 6127 up.go:159] /home/prow/go/src/k8s.io/kops/_rundir/587bf420-5de2-11ed-8af1-9a8d20cfcfb2/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.23.13 --ssh-public-key /tmp/kops-ssh1361984811/key.pub --override cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 35.232.226.60/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-east1-b --master-size e2-standard-2 --project gce-gci-upg-1-4-1-3-ctl-skew I1106 14:52:23.035549 6127 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/587bf420-5de2-11ed-8af1-9a8d20cfcfb2/kops create cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local --cloud gce --kubernetes-version https://storage.googleapis.com/kubernetes-release/release/v1.23.13 --ssh-public-key /tmp/kops-ssh1361984811/key.pub --override cluster.spec.nodePortAccess=0.0.0.0/0 --channel=alpha --networking=calico --container-runtime=containerd --gce-service-account=default --admin-access 35.232.226.60/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-east1-b --master-size e2-standard-2 --project gce-gci-upg-1-4-1-3-ctl-skew I1106 14:52:23.068915 6449 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true I1106 14:52:23.090643 6449 create_cluster.go:834] Using SSH public key: /tmp/kops-ssh1361984811/key.pub I1106 14:52:23.325543 6449 new_cluster.go:530] VMs will be configured to use specified Service Account: default ... skipping 371 lines ... I1106 14:52:30.682758 6471 keypair.go:222] Issuing new certificate: "etcd-manager-ca-events" W1106 14:52:30.683944 6471 vfs_castore.go:382] CA private key was not found I1106 14:52:30.685799 6471 keypair.go:222] Issuing new certificate: "etcd-manager-ca-main" I1106 14:52:30.881318 6471 keypair.go:222] Issuing new certificate: "kubernetes-ca" I1106 14:52:31.074054 6471 keypair.go:222] Issuing new certificate: "service-account" I1106 14:52:43.244176 6471 executor.go:111] Tasks: 42 done / 67 total; 19 can run W1106 14:52:53.099103 6471 executor.go:139] error running task "ForwardingRule/api-e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd-k8s-local" (9m50s remaining to succeed): error creating ForwardingRule "api-e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd-k8s-local": googleapi: Error 400: The resource 'projects/gce-gci-upg-1-4-1-3-ctl-skew/regions/us-east1/targetPools/api-e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd-k8s-local' is not ready, resourceNotReady I1106 14:52:53.099183 6471 executor.go:111] Tasks: 60 done / 67 total; 5 can run I1106 14:52:59.972192 6471 executor.go:111] Tasks: 65 done / 67 total; 2 can run I1106 14:53:11.043988 6471 executor.go:111] Tasks: 67 done / 67 total; 0 can run I1106 14:53:11.098154 6471 update_cluster.go:326] Exporting kubeconfig for cluster kOps has set your kubectl context to e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local ... skipping 8 lines ... I1106 14:53:21.348242 6127 up.go:243] /home/prow/go/src/k8s.io/kops/_rundir/587bf420-5de2-11ed-8af1-9a8d20cfcfb2/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local --count 10 --wait 15m0s I1106 14:53:21.348302 6127 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/587bf420-5de2-11ed-8af1-9a8d20cfcfb2/kops validate cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local --count 10 --wait 15m0s I1106 14:53:21.380335 6491 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true Validating cluster e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local W1106 14:53:51.693535 6491 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.139.190.214/api/v1/nodes": dial tcp 34.139.190.214:443: i/o timeout W1106 14:54:01.727387 6491 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.139.190.214/api/v1/nodes": dial tcp 34.139.190.214:443: connect: connection refused W1106 14:54:11.761106 6491 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.139.190.214/api/v1/nodes": dial tcp 34.139.190.214:443: connect: connection refused W1106 14:54:21.795954 6491 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.139.190.214/api/v1/nodes": dial tcp 34.139.190.214:443: connect: connection refused W1106 14:54:31.830489 6491 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.139.190.214/api/v1/nodes": dial tcp 34.139.190.214:443: connect: connection refused W1106 14:54:41.864647 6491 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.139.190.214/api/v1/nodes": dial tcp 34.139.190.214:443: connect: connection refused W1106 14:54:51.899401 6491 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.139.190.214/api/v1/nodes": dial tcp 34.139.190.214:443: connect: connection refused W1106 14:55:01.934330 6491 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.139.190.214/api/v1/nodes": dial tcp 34.139.190.214:443: connect: connection refused W1106 14:55:11.967938 6491 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.139.190.214/api/v1/nodes": dial tcp 34.139.190.214:443: connect: connection refused W1106 14:55:22.003351 6491 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.139.190.214/api/v1/nodes": dial tcp 34.139.190.214:443: connect: connection refused W1106 14:55:32.036847 6491 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.139.190.214/api/v1/nodes": dial tcp 34.139.190.214:443: connect: connection refused W1106 14:55:42.069979 6491 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.139.190.214/api/v1/nodes": dial tcp 34.139.190.214:443: connect: connection refused W1106 14:56:02.104401 6491 validate_cluster.go:184] (will retry): unexpected error during validation: error listing nodes: Get "https://34.139.190.214/api/v1/nodes": net/http: TLS handshake timeout I1106 14:56:12.490927 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 5 lines ... Machine https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/master-us-east1-b-3mnk machine "https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/master-us-east1-b-3mnk" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-1wht machine "https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-1wht" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-56mw machine "https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-56mw" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-f836 machine "https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-f836" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-qqjd machine "https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-qqjd" has not yet joined cluster Validation Failed W1106 14:56:13.472755 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:56:23.857937 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... Machine https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-1wht machine "https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-1wht" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-56mw machine "https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-56mw" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-f836 machine "https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-f836" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-qqjd machine "https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-qqjd" has not yet joined cluster Node master-us-east1-b-3mnk node "master-us-east1-b-3mnk" of role "master" is not ready Validation Failed W1106 14:56:24.621853 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:56:34.945573 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... Machine https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-1wht machine "https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-1wht" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-56mw machine "https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-56mw" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-f836 machine "https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-f836" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-qqjd machine "https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-qqjd" has not yet joined cluster Node master-us-east1-b-3mnk node "master-us-east1-b-3mnk" of role "master" is not ready Validation Failed W1106 14:56:35.701915 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:56:46.024167 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 13 lines ... Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is pending Pod kube-system/coredns-5dbfc5fddc-75bd9 system-cluster-critical pod "coredns-5dbfc5fddc-75bd9" is pending Pod kube-system/coredns-autoscaler-6bc4c99775-cb6b5 system-cluster-critical pod "coredns-autoscaler-6bc4c99775-cb6b5" is pending Pod kube-system/dns-controller-6b8c58c6cf-bzg8k system-cluster-critical pod "dns-controller-6b8c58c6cf-bzg8k" is pending Pod kube-system/kube-scheduler-master-us-east1-b-3mnk system-cluster-critical pod "kube-scheduler-master-us-east1-b-3mnk" is pending Validation Failed W1106 14:56:47.155224 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:56:57.520480 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 14 lines ... Pod kube-system/coredns-5dbfc5fddc-75bd9 system-cluster-critical pod "coredns-5dbfc5fddc-75bd9" is pending Pod kube-system/coredns-autoscaler-6bc4c99775-cb6b5 system-cluster-critical pod "coredns-autoscaler-6bc4c99775-cb6b5" is pending Pod kube-system/dns-controller-6b8c58c6cf-bzg8k system-cluster-critical pod "dns-controller-6b8c58c6cf-bzg8k" is pending Pod kube-system/etcd-manager-main-master-us-east1-b-3mnk system-cluster-critical pod "etcd-manager-main-master-us-east1-b-3mnk" is pending Pod kube-system/kops-controller-96c77 system-cluster-critical pod "kops-controller-96c77" is pending Validation Failed W1106 14:56:58.517896 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:57:06.873536 6127 boskos.go:86] Sending heartbeat to Boskos I1106 14:57:08.883480 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 ... skipping 13 lines ... Pod kube-system/calico-kube-controllers-7f8977dffb-rgwkz system-cluster-critical pod "calico-kube-controllers-7f8977dffb-rgwkz" is pending Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is pending Pod kube-system/coredns-5dbfc5fddc-75bd9 system-cluster-critical pod "coredns-5dbfc5fddc-75bd9" is pending Pod kube-system/coredns-autoscaler-6bc4c99775-cb6b5 system-cluster-critical pod "coredns-autoscaler-6bc4c99775-cb6b5" is pending Pod kube-system/dns-controller-6b8c58c6cf-bzg8k system-cluster-critical pod "dns-controller-6b8c58c6cf-bzg8k" is pending Validation Failed W1106 14:57:09.666168 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:57:20.015214 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 14 lines ... Pod kube-system/coredns-5dbfc5fddc-75bd9 system-cluster-critical pod "coredns-5dbfc5fddc-75bd9" is pending Pod kube-system/coredns-autoscaler-6bc4c99775-cb6b5 system-cluster-critical pod "coredns-autoscaler-6bc4c99775-cb6b5" is pending Pod kube-system/dns-controller-6b8c58c6cf-bzg8k system-cluster-critical pod "dns-controller-6b8c58c6cf-bzg8k" is pending Pod kube-system/kube-apiserver-master-us-east1-b-3mnk system-cluster-critical pod "kube-apiserver-master-us-east1-b-3mnk" is pending Pod kube-system/kube-proxy-master-us-east1-b-3mnk system-node-critical pod "kube-proxy-master-us-east1-b-3mnk" is pending Validation Failed W1106 14:57:20.798765 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:57:31.122158 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 12 lines ... Pod kube-system/calico-kube-controllers-7f8977dffb-rgwkz system-cluster-critical pod "calico-kube-controllers-7f8977dffb-rgwkz" is pending Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is pending Pod kube-system/coredns-5dbfc5fddc-75bd9 system-cluster-critical pod "coredns-5dbfc5fddc-75bd9" is pending Pod kube-system/coredns-autoscaler-6bc4c99775-cb6b5 system-cluster-critical pod "coredns-autoscaler-6bc4c99775-cb6b5" is pending Pod kube-system/dns-controller-6b8c58c6cf-bzg8k system-cluster-critical pod "dns-controller-6b8c58c6cf-bzg8k" is pending Validation Failed W1106 14:57:31.870109 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:57:42.182509 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 9 lines ... Machine https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-f836 machine "https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-f836" has not yet joined cluster Machine https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-qqjd machine "https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-4-1-3-ctl-skew/zones/us-east1-b/instances/nodes-us-east1-b-qqjd" has not yet joined cluster Node master-us-east1-b-3mnk master "master-us-east1-b-3mnk" is missing kube-controller-manager pod Pod kube-system/coredns-5dbfc5fddc-75bd9 system-cluster-critical pod "coredns-5dbfc5fddc-75bd9" is pending Pod kube-system/coredns-autoscaler-6bc4c99775-cb6b5 system-cluster-critical pod "coredns-autoscaler-6bc4c99775-cb6b5" is pending Validation Failed W1106 14:57:42.928357 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:57:53.282956 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 11 lines ... Node nodes-us-east1-b-f836 node "nodes-us-east1-b-f836" of role "node" is not ready Pod kube-system/calico-node-kbfss system-node-critical pod "calico-node-kbfss" is pending Pod kube-system/coredns-5dbfc5fddc-75bd9 system-cluster-critical pod "coredns-5dbfc5fddc-75bd9" is pending Pod kube-system/coredns-autoscaler-6bc4c99775-cb6b5 system-cluster-critical pod "coredns-autoscaler-6bc4c99775-cb6b5" is pending Pod kube-system/kube-proxy-nodes-us-east1-b-f836 system-node-critical pod "kube-proxy-nodes-us-east1-b-f836" is pending Validation Failed W1106 14:57:54.145969 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:58:04.471465 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 16 lines ... Pod kube-system/calico-node-cjg42 system-node-critical pod "calico-node-cjg42" is pending Pod kube-system/calico-node-kbfss system-node-critical pod "calico-node-kbfss" is pending Pod kube-system/calico-node-zt2bw system-node-critical pod "calico-node-zt2bw" is pending Pod kube-system/coredns-5dbfc5fddc-75bd9 system-cluster-critical pod "coredns-5dbfc5fddc-75bd9" is pending Pod kube-system/coredns-autoscaler-6bc4c99775-cb6b5 system-cluster-critical pod "coredns-autoscaler-6bc4c99775-cb6b5" is pending Validation Failed W1106 14:58:05.129828 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:58:15.441296 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 16 lines ... Pod kube-system/calico-node-cjg42 system-node-critical pod "calico-node-cjg42" is pending Pod kube-system/calico-node-kbfss system-node-critical pod "calico-node-kbfss" is pending Pod kube-system/calico-node-zt2bw system-node-critical pod "calico-node-zt2bw" is pending Pod kube-system/coredns-5dbfc5fddc-75bd9 system-cluster-critical pod "coredns-5dbfc5fddc-75bd9" is pending Pod kube-system/coredns-autoscaler-6bc4c99775-cb6b5 system-cluster-critical pod "coredns-autoscaler-6bc4c99775-cb6b5" is pending Validation Failed W1106 14:58:16.065107 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:58:26.496635 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 12 lines ... Node nodes-us-east1-b-qqjd node "nodes-us-east1-b-qqjd" of role "node" is not ready Pod kube-system/calico-node-6qn64 system-node-critical pod "calico-node-6qn64" is pending Pod kube-system/calico-node-cjg42 system-node-critical pod "calico-node-cjg42" is not ready (calico-node) Pod kube-system/calico-node-kbfss system-node-critical pod "calico-node-kbfss" is not ready (calico-node) Pod kube-system/calico-node-zt2bw system-node-critical pod "calico-node-zt2bw" is pending Validation Failed W1106 14:58:27.238675 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:58:37.597989 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 10 lines ... KIND NAME MESSAGE Pod kube-system/calico-node-6qn64 system-node-critical pod "calico-node-6qn64" is not ready (calico-node) Pod kube-system/calico-node-cjg42 system-node-critical pod "calico-node-cjg42" is not ready (calico-node) Pod kube-system/calico-node-kbfss system-node-critical pod "calico-node-kbfss" is not ready (calico-node) Pod kube-system/calico-node-zt2bw system-node-critical pod "calico-node-zt2bw" is not ready (calico-node) Validation Failed W1106 14:58:38.196575 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:58:48.630421 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 11 lines ... Pod kube-system/calico-node-6qn64 system-node-critical pod "calico-node-6qn64" is not ready (calico-node) Pod kube-system/calico-node-cjg42 system-node-critical pod "calico-node-cjg42" is not ready (calico-node) Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Pod kube-system/calico-node-kbfss system-node-critical pod "calico-node-kbfss" is not ready (calico-node) Pod kube-system/calico-node-zt2bw system-node-critical pod "calico-node-zt2bw" is not ready (calico-node) Validation Failed W1106 14:58:49.410276 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:58:59.733642 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 10 lines ... KIND NAME MESSAGE Pod kube-system/calico-node-6qn64 system-node-critical pod "calico-node-6qn64" is not ready (calico-node) Pod kube-system/calico-node-cjg42 system-node-critical pod "calico-node-cjg42" is not ready (calico-node) Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Pod kube-system/calico-node-zt2bw system-node-critical pod "calico-node-zt2bw" is not ready (calico-node) Validation Failed W1106 14:59:00.477952 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:59:10.931062 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 14:59:11.661738 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:59:22.057818 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 14:59:22.663315 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:59:32.985562 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 14:59:33.738242 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:59:44.100881 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 14:59:44.882864 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 14:59:55.261019 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 14:59:55.888049 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:00:06.198324 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:00:06.984707 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:00:17.337589 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:00:18.094233 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:00:28.527600 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:00:29.151721 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:00:39.532867 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:00:40.095040 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:00:50.418107 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:00:51.235737 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:01:01.599540 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:01:02.220742 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:01:12.586076 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:01:13.319779 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:01:23.747073 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:01:24.489955 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:01:34.941213 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:01:35.854701 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:01:46.131372 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:01:46.890754 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:01:57.221479 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:01:57.969332 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:02:06.884017 6127 boskos.go:86] Sending heartbeat to Boskos I1106 15:02:08.342683 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 ... skipping 8 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:02:09.089184 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:02:19.454633 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:02:20.199608 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:02:30.518182 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:02:31.244936 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:02:41.641849 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:02:42.201608 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:02:52.527940 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:02:53.090882 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:03:03.445209 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:03:04.045484 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:03:14.372943 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:03:15.254588 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:03:25.656902 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:03:26.271242 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:03:36.592937 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:03:37.190134 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:03:47.492227 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:03:48.081490 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:03:58.530079 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:03:59.124173 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:04:09.499171 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:04:10.113258 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:04:20.489404 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:04:21.068976 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:04:31.475061 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:04:32.256725 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:04:42.615088 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:04:43.385621 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:04:53.714300 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:04:54.382810 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:05:04.735049 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:05:05.346085 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:05:15.649388 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:05:16.246824 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:05:26.660073 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:05:27.253134 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:05:37.641259 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:05:38.196758 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:05:48.528785 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:05:49.306734 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:05:59.607236 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:06:00.210444 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:06:10.660960 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:06:11.279008 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:06:21.626334 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:06:22.224875 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:06:32.559769 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:06:33.130953 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:06:43.496436 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:06:44.110318 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:06:54.596640 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:06:55.220013 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:07:05.625084 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:07:06.385805 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:07:06.894361 6127 boskos.go:86] Sending heartbeat to Boskos I1106 15:07:16.819460 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 ... skipping 8 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:07:17.401228 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:07:27.739001 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:07:28.376516 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:07:38.742366 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:07:39.394128 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:07:49.718696 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:07:50.293105 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:08:00.621698 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:08:01.226962 6491 validate_cluster.go:232] (will retry): cluster not yet healthy I1106 15:08:11.601913 6491 gce_cloud.go:295] Scanning zones: [us-east1-b us-east1-c us-east1-d us-east1-a] INSTANCE GROUPS NAME ROLE MACHINETYPE MIN MAX SUBNETS master-us-east1-b Master e2-standard-2 1 1 us-east1 nodes-us-east1-b Node n1-standard-2 4 4 us-east1 ... skipping 7 lines ... nodes-us-east1-b-qqjd node True VALIDATION ERRORS KIND NAME MESSAGE Pod kube-system/calico-node-fw7td system-node-critical pod "calico-node-fw7td" is not ready (calico-node) Validation Failed W1106 15:08:12.225168 6491 validate_cluster.go:232] (will retry): cluster not yet healthy Error: Validation failed: wait time exceeded during validation I1106 15:08:22.229707 6127 dumplogs.go:45] /home/prow/go/src/k8s.io/kops/_rundir/587bf420-5de2-11ed-8af1-9a8d20cfcfb2/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1361984811/key --ssh-user prow I1106 15:08:22.229751 6127 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/587bf420-5de2-11ed-8af1-9a8d20cfcfb2/kops toolbox dump --name e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local --dir /logs/artifacts --private-key /tmp/kops-ssh1361984811/key --ssh-user prow I1106 15:08:42.907733 6127 dumplogs.go:78] /home/prow/go/src/k8s.io/kops/_rundir/587bf420-5de2-11ed-8af1-9a8d20cfcfb2/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local -o yaml I1106 15:08:42.907775 6127 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/587bf420-5de2-11ed-8af1-9a8d20cfcfb2/kops get cluster --name e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local -o yaml I1106 15:08:43.039052 6127 dumplogs.go:78] /home/prow/go/src/k8s.io/kops/_rundir/587bf420-5de2-11ed-8af1-9a8d20cfcfb2/kops get instancegroups --name e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local -o yaml I1106 15:08:43.039089 6127 local.go:42] ⚙️ /home/prow/go/src/k8s.io/kops/_rundir/587bf420-5de2-11ed-8af1-9a8d20cfcfb2/kops get instancegroups --name e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local -o yaml ... skipping 133 lines ... Deleted cluster: "e2e-e2e-kops-grid-gce-calico-u2004-k23-containerd.k8s.local" I1106 15:10:41.748410 6127 gcs.go:88] gsutil -u gce-gci-upg-1-4-1-3-ctl-skew rm -r gs://gce-gci-upg-1-4-1-3-ctl-skew-state-58 I1106 15:10:41.748470 6127 local.go:42] ⚙️ gsutil -u gce-gci-upg-1-4-1-3-ctl-skew rm -r gs://gce-gci-upg-1-4-1-3-ctl-skew-state-58 Removing gs://gce-gci-upg-1-4-1-3-ctl-skew-state-58/... I1106 15:10:44.270067 6127 down.go:62] releasing boskos project I1106 15:10:44.280240 6127 boskos.go:83] Boskos heartbeat func received signal to close Error: exit status 1 + EXIT_VALUE=1 + set +o xtrace