ci-kubernetes-e2e-gke-gci-stable2-gci-stable1-upgrade-master #1193684693095026691
Recent runs || View in Spyglass
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 1 failed / 6 succeeded |
| Started | |
| Elapsed | 29m47s |
| Revision | |
| Builder | gke-prow-ssd-pool-1a225945-xlz0 |
| links | {u'resultstore': {u'url': u'https://source.cloud.google.com/results/invocations/0b053684-b070-4a50-962e-6f040d697ac5/targets/test'}} |
| pod | e480a7c1-0418-11ea-88c9-c26ecbf388b3 |
| resultstore | https://source.cloud.google.com/results/invocations/0b053684-b070-4a50-962e-6f040d697ac5/targets/test |
| infra-commit | 830666073 |
| job-version | v1.15.6-beta.0.38+7015f71e75f670 |
| pod | e480a7c1-0418-11ea-88c9-c26ecbf388b3 |
| revision | v1.15.6-beta.0.38+7015f71e75f670 |
error creating cluster: error during gcloud container clusters create --quiet --enable-legacy-authorization --project=kubernetes-gci-gke-ingress-1-2 --zone=us-central1-c --machine-type=n1-standard-2 --image-type=gci --num-nodes=3 --network=bootstrap-e2e --cluster-version=1.15.6-beta.0.38+7015f71e75f670 bootstrap-e2e: exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
Deferred TearDown
DumpClusterLogs (--up failed)
Extract
TearDown Previous
Timeout
listResources Before
... skipping 15 lines ... I1111 00:20:04.368] process 45 exited with code 0 after 0.0m I1111 00:20:04.368] Will upload results to gs://kubernetes-jenkins/logs using pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com I1111 00:20:04.369] Root: /workspace I1111 00:20:04.369] cd to /workspace I1111 00:20:04.369] Configure environment... I1111 00:20:04.369] Call: git show -s --format=format:%ct HEAD W1111 00:20:04.375] fatal: not a git repository (or any of the parent directories): .git I1111 00:20:04.376] process 57 exited with code 128 after 0.0m W1111 00:20:04.376] Unable to print commit date for HEAD I1111 00:20:04.377] Call: gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json W1111 00:20:05.041] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com] I1111 00:20:05.419] process 58 exited with code 0 after 0.0m I1111 00:20:05.420] Call: gcloud config get-value account ... skipping 481 lines ... W1111 00:22:50.955] WARNING: Newly created clusters and node-pools will have node auto-upgrade enabled by default. This can be disabled using the `--no-enable-autoupgrade` flag. W1111 00:22:50.956] WARNING: Starting in 1.12, default node pools in new clusters will have their legacy Compute Engine instance metadata endpoints disabled by default. To create a cluster with legacy instance metadata endpoints disabled in the default node pool, run `clusters create` with the flag `--metadata disable-legacy-endpoints=true`. W1111 00:22:50.956] WARNING: Your Pod address range (`--cluster-ipv4-cidr`) can accommodate at most 1008 node(s). W1111 00:22:50.957] This will disable the autorepair feature for nodes. Please see https://cloud.google.com/kubernetes-engine/docs/node-auto-repair for more information on node autorepairs. W1111 00:22:54.173] Creating cluster bootstrap-e2e in us-central1-c... W1111 00:33:27.950] ..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done. W1111 00:33:28.132] ERROR: (gcloud.container.clusters.create) Operation [<Operation W1111 00:33:28.132] clusterConditions: [] W1111 00:33:28.132] detail: u'All cluster resources were brought up, but the cluster API is reporting that: component "kube-apiserver" from endpoint "gke-4b67981a47910482d762-66b5" is unhealthy.' W1111 00:33:28.132] endTime: u'2019-11-11T00:33:24.490577535Z' W1111 00:33:28.132] name: u'operation-1573431774110-8222e55c' W1111 00:33:28.133] nodepoolConditions: [] W1111 00:33:28.133] operationType: OperationTypeValueValuesEnum(CREATE_CLUSTER, 1) W1111 00:33:28.133] selfLink: u'https://test-container.sandbox.googleapis.com/v1/projects/804181854823/zones/us-central1-c/operations/operation-1573431774110-8222e55c' W1111 00:33:28.133] startTime: u'2019-11-11T00:22:54.110895913Z' W1111 00:33:28.133] status: StatusValueValuesEnum(DONE, 3) W1111 00:33:28.133] statusMessage: u'All cluster resources were brought up, but the cluster API is reporting that: component "kube-apiserver" from endpoint "gke-4b67981a47910482d762-66b5" is unhealthy.' W1111 00:33:28.133] targetLink: u'https://test-container.sandbox.googleapis.com/v1/projects/804181854823/zones/us-central1-c/clusters/bootstrap-e2e' W1111 00:33:28.134] zone: u'us-central1-c'>] finished with error: All cluster resources were brought up, but the cluster API is reporting that: component "kube-apiserver" from endpoint "gke-4b67981a47910482d762-66b5" is unhealthy. W1111 00:33:28.196] 2019/11/11 00:33:28 process.go:155: Step 'gcloud container clusters create --quiet --enable-legacy-authorization --project=kubernetes-gci-gke-ingress-1-2 --zone=us-central1-c --machine-type=n1-standard-2 --image-type=gci --num-nodes=3 --network=bootstrap-e2e --cluster-version=1.15.6-beta.0.38+7015f71e75f670 bootstrap-e2e' finished in 10m37.615016118s W1111 00:33:28.788] 2019/11/11 00:33:28 process.go:153: Running: bash -c W1111 00:33:28.788] function log_dump_custom_get_instances() { W1111 00:33:28.788] if [[ $1 == "master" ]]; then W1111 00:33:28.788] return 0 W1111 00:33:28.788] fi ... skipping 14 lines ... I1111 00:33:28.891] Dumping logs from nodes locally to '/workspace/_artifacts' I1111 00:33:28.891] Dumping logs for nodes provided by log_dump_custom_get_instances() function W1111 00:33:30.190] WARNING: --filter : operator evaluation is changing for consistency across Google APIs. metadata.created-by:*zones/us-central1-c/instanceGroupManagers/gke-bootstrap-e2e-default-pool-4ebe18ac-grp currently matches but will not match in the near future. Run `gcloud topic filters` for details. I1111 00:40:34.108] Changing logfiles to be world-readable for download I1111 00:40:34.129] Changing logfiles to be world-readable for download I1111 00:40:34.483] Changing logfiles to be world-readable for download W1111 00:41:05.212] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:41:05.258] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:41:05.549] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:41:41.347] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:41:41.413] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:41:41.628] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:42:17.523] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:42:17.557] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:42:17.739] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:42:53.807] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:42:53.846] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:42:53.916] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:43:29.911] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:43:29.912] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:43:29.940] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:44:06.097] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:44:06.101] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. W1111 00:44:06.144] ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. I1111 00:44:06.245] Copying 'kern.log kube-proxy.log fluentd.log node-problem-detector.log kubelet.cov docker/log kubelet.log supervisor/supervisord.log supervisor/kubelet-stdout.log supervisor/kubelet-stderr.log supervisor/docker-stdout.log supervisor/docker-stderr.log' from gke-bootstrap-e2e-default-pool-4ebe18ac-77jp I1111 00:44:06.247] Copying 'kern.log kube-proxy.log fluentd.log node-problem-detector.log kubelet.cov docker/log kubelet.log supervisor/supervisord.log supervisor/kubelet-stdout.log supervisor/kubelet-stderr.log supervisor/docker-stdout.log supervisor/docker-stderr.log' from gke-bootstrap-e2e-default-pool-4ebe18ac-8778 I1111 00:44:06.248] Copying 'kern.log kube-proxy.log fluentd.log node-problem-detector.log kubelet.cov docker/log kubelet.log supervisor/supervisord.log supervisor/kubelet-stdout.log supervisor/kubelet-stderr.log supervisor/docker-stdout.log supervisor/docker-stderr.log' from gke-bootstrap-e2e-default-pool-4ebe18ac-lzmm W1111 00:44:07.061] W1111 00:44:07.061] Specify --start=46076 in the next get-serial-port-output invocation to get only the new output starting from here. W1111 00:44:07.199] W1111 00:44:07.199] Specify --start=46220 in the next get-serial-port-output invocation to get only the new output starting from here. W1111 00:44:07.221] W1111 00:44:07.222] Specify --start=46175 in the next get-serial-port-output invocation to get only the new output starting from here. W1111 00:46:18.041] ssh: connect to host 34.69.65.167 port 22: Connection timed out W1111 00:46:18.041] ssh: connect to host 35.232.202.19 port 22: Connection timed out W1111 00:46:18.042] ssh: connect to host 35.222.119.59 port 22: Connection timed out W1111 00:46:18.049] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1]. W1111 00:46:18.050] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1]. W1111 00:46:18.050] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1]. W1111 00:46:18.146] Project: kubernetes-gci-gke-ingress-1-2 W1111 00:46:18.147] Network Project: kubernetes-gci-gke-ingress-1-2 W1111 00:46:18.147] Zone: us-central1-c W1111 00:46:20.936] INSTANCE_GROUPS= W1111 00:46:20.937] NODE_NAMES= W1111 00:46:20.937] 2019/11/11 00:46:20 process.go:155: Step 'bash -c ... skipping 21 lines ... W1111 00:48:58.621] 2019/11/11 00:48:58 process.go:155: Step 'gcloud compute firewall-rules describe e2e-ports-4ebe18ac --project=kubernetes-gci-gke-ingress-1-2 --format=value(name)' finished in 781.569856ms W1111 00:48:58.621] 2019/11/11 00:48:58 gke.go:631: Found no rules for firewall 'e2e-ports-4ebe18ac', assuming resources are clean W1111 00:48:59.464] 2019/11/11 00:48:59 process.go:153: Running: gcloud compute networks delete -q bootstrap-e2e --project=kubernetes-gci-gke-ingress-1-2 W1111 00:49:37.320] Deleted [https://www.googleapis.com/compute/v1/projects/kubernetes-gci-gke-ingress-1-2/global/networks/bootstrap-e2e]. W1111 00:49:37.412] 2019/11/11 00:49:37 process.go:155: Step 'gcloud compute networks delete -q bootstrap-e2e --project=kubernetes-gci-gke-ingress-1-2' finished in 37.946228303s W1111 00:49:37.412] 2019/11/11 00:49:37 process.go:96: Saved XML output to /workspace/_artifacts/junit_runner.xml. W1111 00:49:43.684] 2019/11/11 00:49:43 main.go:319: Something went wrong: starting e2e cluster: error creating cluster: error during gcloud container clusters create --quiet --enable-legacy-authorization --project=kubernetes-gci-gke-ingress-1-2 --zone=us-central1-c --machine-type=n1-standard-2 --image-type=gci --num-nodes=3 --network=bootstrap-e2e --cluster-version=1.15.6-beta.0.38+7015f71e75f670 bootstrap-e2e: exit status 1 W1111 00:49:43.689] Traceback (most recent call last): W1111 00:49:43.689] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 778, in <module> W1111 00:49:43.689] main(parse_args()) W1111 00:49:43.690] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 626, in main W1111 00:49:43.690] mode.start(runner_args) W1111 00:49:43.690] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 262, in start W1111 00:49:43.690] check_env(env, self.command, *args) W1111 00:49:43.690] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 111, in check_env W1111 00:49:43.690] subprocess.check_call(cmd, env=env) W1111 00:49:43.691] File "/usr/lib/python2.7/subprocess.py", line 190, in check_call W1111 00:49:43.691] raise CalledProcessError(retcode, cmd) W1111 00:49:43.692] subprocess.CalledProcessError: Command '('kubetest', '--dump=/workspace/_artifacts', '--gcp-service-account=/etc/service-account/service-account.json', '--up', '--down', '--test', '--deployment=gke', '--provider=gke', '--cluster=bootstrap-e2e', '--gcp-network=bootstrap-e2e', '--check-leaked-resources', '--check-version-skew=false', '--extract=ci/k8s-stable1', '--extract=ci/k8s-stable2', '--gcp-cloud-sdk=gs://cloud-sdk-testing/ci/staging', '--gcp-node-image=gci', '--gcp-zone=us-central1-c', '--gke-create-command=container clusters create --quiet --enable-legacy-authorization', '--gke-environment=test', '--test_args=--kubectl-path=../../../../kubernetes_skew/cluster/kubectl.sh --minStartupPods=8', '--timeout=900m', '--upgrade_args=--ginkgo.focus=\\[Feature:MasterUpgrade\\] --upgrade-target=ci/k8s-stable1 --upgrade-image=gci')' returned non-zero exit status 1 E1111 00:49:43.699] Command failed I1111 00:49:43.699] process 262 exited with code 1 after 29.6m E1111 00:49:43.699] FAIL: ci-kubernetes-e2e-gke-gci-stable2-gci-stable1-upgrade-master I1111 00:49:43.700] Call: gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json W1111 00:49:44.236] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com] I1111 00:49:44.302] process 2507 exited with code 0 after 0.0m I1111 00:49:44.302] Call: gcloud config get-value account I1111 00:49:44.625] process 2519 exited with code 0 after 0.0m I1111 00:49:44.625] Will upload results to gs://kubernetes-jenkins/logs using pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com I1111 00:49:44.626] Upload result and artifacts... I1111 00:49:44.626] Gubernator results at https://gubernator.k8s.io/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-gci-stable2-gci-stable1-upgrade-master/1193684693095026691 I1111 00:49:44.626] Call: gsutil ls gs://kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-gci-stable2-gci-stable1-upgrade-master/1193684693095026691/artifacts W1111 00:49:45.481] CommandException: One or more URLs matched no objects. E1111 00:49:45.598] Command failed I1111 00:49:45.598] process 2531 exited with code 1 after 0.0m W1111 00:49:45.598] Remote dir gs://kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-gci-stable2-gci-stable1-upgrade-master/1193684693095026691/artifacts not exist yet I1111 00:49:45.598] Call: gsutil -m -q -o GSUtil:use_magicfile=True cp -r -c -z log,txt,xml /workspace/_artifacts gs://kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-gci-stable2-gci-stable1-upgrade-master/1193684693095026691/artifacts I1111 00:49:47.332] process 2675 exited with code 0 after 0.0m I1111 00:49:47.333] Call: git rev-parse HEAD W1111 00:49:47.336] fatal: not a git repository (or any of the parent directories): .git E1111 00:49:47.336] Command failed I1111 00:49:47.337] process 3210 exited with code 128 after 0.0m I1111 00:49:47.337] Call: git rev-parse HEAD I1111 00:49:47.340] process 3211 exited with code 0 after 0.0m I1111 00:49:47.340] Call: gsutil stat gs://kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-gci-stable2-gci-stable1-upgrade-master/jobResultsCache.json I1111 00:49:48.195] process 3212 exited with code 0 after 0.0m I1111 00:49:48.196] Call: gsutil -q cat 'gs://kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-gci-stable2-gci-stable1-upgrade-master/jobResultsCache.json#1573390178690633' ... skipping 8 lines ...