This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 1 failed / 23 succeeded
Started2019-05-18 12:24
Elapsed59m40s
Revision
Buildergke-prow-containerd-pool-99179761-1b0c
links{u'resultstore': {u'url': u'https://source.cloud.google.com/results/invocations/30ad39a9-f33d-4d80-91d7-1935c683f89d/targets/test'}}
podbd201e49-7967-11e9-9444-660052217703
resultstorehttps://source.cloud.google.com/results/invocations/30ad39a9-f33d-4d80-91d7-1935c683f89d/targets/test
infra-commit95ef25d87
job-versionv1.12.9-beta.0.48+3e39ad05dbde34
master_os_image
node_os_imageubuntu-gke-1804-d1809-0-v20190517
podbd201e49-7967-11e9-9444-660052217703
revisionv1.12.9-beta.0.48+3e39ad05dbde34

Test Failures


Check APIReachability 40s

error starting ./cluster/kubectl.sh --match-server-version=false version: exec: already started
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 23 Passed Tests

Show 2006 Skipped Tests

Error lines from build-log.txt

... skipping 12 lines ...
I0518 12:24:28.544] process 44 exited with code 0 after 0.0m
I0518 12:24:28.545] Will upload results to gs://kubernetes-jenkins/logs using pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com
I0518 12:24:28.545] Root: /workspace
I0518 12:24:28.545] cd to /workspace
I0518 12:24:28.545] Configure environment...
I0518 12:24:28.546] Call:  git show -s --format=format:%ct HEAD
W0518 12:24:28.550] fatal: Not a git repository (or any of the parent directories): .git
I0518 12:24:28.550] process 56 exited with code 128 after 0.0m
W0518 12:24:28.550] Unable to print commit date for HEAD
I0518 12:24:28.551] Call:  gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json
W0518 12:24:29.152] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com]
I0518 12:24:29.539] process 57 exited with code 0 after 0.0m
I0518 12:24:29.540] Call:  gcloud config get-value account
... skipping 418 lines ...
W0518 12:32:09.868] 2019/05/18 12:32:09 process.go:155: Step 'gcloud container clusters create --quiet --project=gke-gci-upg-1-4-1-3-ctl-skew --zone=us-west1-b --machine-type=n1-standard-2 --image-type=custom --num-nodes=3 --network=test-0101dc0a7b --image-family=pipeline-2 --image-project=ubuntu-os-gke-cloud-devel --cluster-version=1.12.9-beta.0.48+3e39ad05dbde34 test-0101dc0a7b' finished in 3m48.899706845s
W0518 12:32:09.868] 2019/05/18 12:32:09 process.go:153: Running: ./cluster/kubectl.sh --match-server-version=false version
I0518 12:32:09.969] NAME             LOCATION    MASTER_VERSION                   MASTER_IP       MACHINE_TYPE   NODE_VERSION                     NUM_NODES  STATUS
I0518 12:32:09.969] test-0101dc0a7b  us-west1-b  1.12.9-beta.0.48+3e39ad05dbde34  35.233.141.113  n1-standard-2  1.12.9-beta.0.48+3e39ad05dbde34  3          RUNNING
W0518 12:32:10.482] The connection to the server 35.233.141.113 was refused - did you specify the right host or port?
W0518 12:32:10.486] 2019/05/18 12:32:10 process.go:155: Step './cluster/kubectl.sh --match-server-version=false version' finished in 618.28167ms
W0518 12:32:10.486] 2019/05/18 12:32:10 e2e.go:333: Failed to reach api. Sleeping for 10 seconds before retrying... ([./cluster/kubectl.sh --match-server-version=false version])
W0518 12:32:20.486] 2019/05/18 12:32:20 process.go:153: Running: ./cluster/kubectl.sh --match-server-version=false version
W0518 12:32:20.487] 2019/05/18 12:32:20 process.go:155: Step './cluster/kubectl.sh --match-server-version=false version' finished in 16.718µs
W0518 12:32:20.487] 2019/05/18 12:32:20 e2e.go:333: Failed to reach api. Sleeping for 10 seconds before retrying... ([./cluster/kubectl.sh --match-server-version=false version])
W0518 12:32:30.487] 2019/05/18 12:32:30 process.go:153: Running: ./cluster/kubectl.sh --match-server-version=false version
W0518 12:32:30.487] 2019/05/18 12:32:30 process.go:155: Step './cluster/kubectl.sh --match-server-version=false version' finished in 12.87µs
W0518 12:32:30.487] 2019/05/18 12:32:30 e2e.go:333: Failed to reach api. Sleeping for 10 seconds before retrying... ([./cluster/kubectl.sh --match-server-version=false version])
W0518 12:32:40.489] 2019/05/18 12:32:40 process.go:153: Running: ./cluster/kubectl.sh --match-server-version=false version
W0518 12:32:40.490] 2019/05/18 12:32:40 process.go:155: Step './cluster/kubectl.sh --match-server-version=false version' finished in 12.77µs
W0518 12:32:40.490] 2019/05/18 12:32:40 e2e.go:333: Failed to reach api. Sleeping for 10 seconds before retrying... ([./cluster/kubectl.sh --match-server-version=false version])
W0518 12:32:50.487] 2019/05/18 12:32:50 process.go:153: Running: ./cluster/kubectl.sh --match-server-version=false version
W0518 12:32:50.489] 2019/05/18 12:32:50 process.go:155: Step './cluster/kubectl.sh --match-server-version=false version' finished in 16.652µs
W0518 12:32:50.490] 2019/05/18 12:32:50 process.go:153: Running: ./cluster/kubectl.sh --match-server-version=false get nodes -oyaml
W0518 12:32:52.197] 2019/05/18 12:32:52 process.go:155: Step './cluster/kubectl.sh --match-server-version=false get nodes -oyaml' finished in 1.709363964s
W0518 12:32:52.197] 2019/05/18 12:32:52 e2e.go:444: Listing resources...
W0518 12:32:52.198] 2019/05/18 12:32:52 process.go:153: Running: ./cluster/gce/list-resources.sh
... skipping 45 lines ...
I0518 12:33:27.485] W0518 12:33:27.485553    1745 gce.go:467] No network name or URL specified.
I0518 12:33:30.301] May 18 12:33:30.300: INFO: cluster-master-image: 
I0518 12:33:30.301] May 18 12:33:30.300: INFO: cluster-node-image: ubuntu-gke-1804-d1809-0-v20190517
I0518 12:33:30.301] May 18 12:33:30.300: INFO: >>> kubeConfig: /tmp/gke-kubecfg993642979
I0518 12:33:30.303] May 18 12:33:30.303: INFO: Waiting up to 30m0s for all (but 0) nodes to be schedulable
I0518 12:33:30.583] May 18 12:33:30.582: INFO: Waiting up to 10m0s for all pods (need at least 8) in namespace 'kube-system' to be running and ready
I0518 12:33:31.146] May 18 12:33:31.145: INFO: The status of Pod fluentd-gcp-v3.2.0-js4ms is Running (Ready = false), waiting for it to be either Running (with Ready = true) or Failed
I0518 12:33:31.146] May 18 12:33:31.146: INFO: The status of Pod kube-dns-695b668597-5czsz is Running (Ready = false), waiting for it to be either Running (with Ready = true) or Failed
I0518 12:33:31.147] May 18 12:33:31.146: INFO: 15 / 17 pods in namespace 'kube-system' are running and ready (0 seconds elapsed)
I0518 12:33:31.147] May 18 12:33:31.146: INFO: expected 8 pod replicas in namespace 'kube-system', 7 are Running and Ready.
I0518 12:33:31.147] May 18 12:33:31.146: INFO: POD                        NODE                                            PHASE    GRACE  CONDITIONS
I0518 12:33:31.148] May 18 12:33:31.146: INFO: fluentd-gcp-v3.2.0-js4ms   gke-test-0101dc0a7b-default-pool-9ace6e89-6r5g  Running  60s    [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:32:04 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:29 +0000 UTC ContainersNotReady containers with unready status: [fluentd-gcp prometheus-to-sd-exporter]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:29 +0000 UTC ContainersNotReady containers with unready status: [fluentd-gcp prometheus-to-sd-exporter]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:32:04 +0000 UTC  }]
I0518 12:33:31.148] May 18 12:33:31.147: INFO: kube-dns-695b668597-5czsz  gke-test-0101dc0a7b-default-pool-9ace6e89-r20t  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:18 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:18 +0000 UTC ContainersNotReady containers with unready status: [kubedns]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:18 +0000 UTC ContainersNotReady containers with unready status: [kubedns]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:18 +0000 UTC  }]
I0518 12:33:31.149] May 18 12:33:31.147: INFO: 
I0518 12:33:33.282] May 18 12:33:33.282: INFO: The status of Pod fluentd-gcp-v3.2.0-js4ms is Running (Ready = false), waiting for it to be either Running (with Ready = true) or Failed
I0518 12:33:33.283] May 18 12:33:33.282: INFO: The status of Pod kube-dns-695b668597-5czsz is Running (Ready = false), waiting for it to be either Running (with Ready = true) or Failed
I0518 12:33:33.283] May 18 12:33:33.283: INFO: 15 / 17 pods in namespace 'kube-system' are running and ready (2 seconds elapsed)
I0518 12:33:33.283] May 18 12:33:33.283: INFO: expected 8 pod replicas in namespace 'kube-system', 7 are Running and Ready.
I0518 12:33:33.283] May 18 12:33:33.283: INFO: POD                        NODE                                            PHASE    GRACE  CONDITIONS
I0518 12:33:33.284] May 18 12:33:33.284: INFO: fluentd-gcp-v3.2.0-js4ms   gke-test-0101dc0a7b-default-pool-9ace6e89-6r5g  Running  60s    [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:32:04 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:29 +0000 UTC ContainersNotReady containers with unready status: [fluentd-gcp prometheus-to-sd-exporter]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:29 +0000 UTC ContainersNotReady containers with unready status: [fluentd-gcp prometheus-to-sd-exporter]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:32:04 +0000 UTC  }]
I0518 12:33:33.285] May 18 12:33:33.284: INFO: kube-dns-695b668597-5czsz  gke-test-0101dc0a7b-default-pool-9ace6e89-r20t  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:18 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:18 +0000 UTC ContainersNotReady containers with unready status: [kubedns]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:18 +0000 UTC ContainersNotReady containers with unready status: [kubedns]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:18 +0000 UTC  }]
I0518 12:33:33.285] May 18 12:33:33.285: INFO: 
I0518 12:33:35.285] May 18 12:33:35.285: INFO: The status of Pod fluentd-gcp-v3.2.0-q9mwh is Pending (Ready = false), waiting for it to be either Running (with Ready = true) or Failed
I0518 12:33:35.286] May 18 12:33:35.286: INFO: The status of Pod kube-dns-695b668597-5czsz is Running (Ready = false), waiting for it to be either Running (with Ready = true) or Failed
I0518 12:33:35.286] May 18 12:33:35.286: INFO: 15 / 17 pods in namespace 'kube-system' are running and ready (4 seconds elapsed)
I0518 12:33:35.286] May 18 12:33:35.286: INFO: expected 8 pod replicas in namespace 'kube-system', 7 are Running and Ready.
I0518 12:33:35.287] May 18 12:33:35.286: INFO: POD                        NODE                                            PHASE    GRACE  CONDITIONS
I0518 12:33:35.287] May 18 12:33:35.286: INFO: fluentd-gcp-v3.2.0-q9mwh   gke-test-0101dc0a7b-default-pool-9ace6e89-6r5g  Pending         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:34 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:34 +0000 UTC ContainersNotReady containers with unready status: [fluentd-gcp prometheus-to-sd-exporter]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:34 +0000 UTC ContainersNotReady containers with unready status: [fluentd-gcp prometheus-to-sd-exporter]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:34 +0000 UTC  }]
I0518 12:33:35.287] May 18 12:33:35.286: INFO: kube-dns-695b668597-5czsz  gke-test-0101dc0a7b-default-pool-9ace6e89-r20t  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:18 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:18 +0000 UTC ContainersNotReady containers with unready status: [kubedns]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:18 +0000 UTC ContainersNotReady containers with unready status: [kubedns]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2019-05-18 12:33:18 +0000 UTC  }]
I0518 12:33:35.288] May 18 12:33:35.286: INFO: 
... skipping 2183 lines ...
I0518 13:18:32.179] ------------------------------
I0518 13:18:32.180] SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSMay 18 13:18:32.178: INFO: Running AfterSuite actions on all node
I0518 13:18:32.180] May 18 13:18:32.178: INFO: Running AfterSuite actions on node 1
I0518 13:18:32.180] May 18 13:18:32.178: INFO: Skipping dumping logs from cluster
I0518 13:18:32.180] 
I0518 13:18:32.181] Ran 6 of 2012 Specs in 2704.890 seconds
I0518 13:18:32.194] SUCCESS! -- 6 Passed | 0 Failed | 0 Pending | 2006 Skipped PASS
I0518 13:18:32.204] 
I0518 13:18:32.206] Ginkgo ran 1 suite in 45m5.783793176s
I0518 13:18:32.206] Test Suite Passed
I0518 13:18:32.212] Checking for custom logdump instances, if any
I0518 13:18:32.218] Using 'use_custom_instance_list' with gke, skipping check for LOG_DUMP_SSH_KEY and LOG_DUMP_SSH_USER
I0518 13:18:32.260] Dumping logs from master locally to '/workspace/_artifacts'
... skipping 45 lines ...
W0518 13:19:19.529] #          See https://cloud.google.com/kubernetes-engine/docs/concepts/node-images#modifications
W0518 13:19:19.529] #          for more information.
W0518 13:19:19.529] ##############################################################################
W0518 13:19:19.799] scp: /var/log/fluentd.log*: No such file or directory
W0518 13:19:19.800] scp: /var/log/node-problem-detector.log*: No such file or directory
W0518 13:19:19.800] scp: /var/log/kubelet.cov*: No such file or directory
W0518 13:19:19.803] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1].
W0518 13:19:20.122] scp: /var/log/fluentd.log*: No such file or directory
W0518 13:19:20.123] scp: /var/log/node-problem-detector.log*: No such file or directory
W0518 13:19:20.123] scp: /var/log/kubelet.cov*: No such file or directory
W0518 13:19:20.130] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1].
W0518 13:19:20.484] ##############################################################################
W0518 13:19:20.484] # WARNING: Any changes on the boot disk of the node must be made via
W0518 13:19:20.484] #          DaemonSet in order to preserve them across node (re)creations.
W0518 13:19:20.484] #          Node will be (re)created during manual-upgrade, auto-upgrade,
W0518 13:19:20.485] #          auto-repair or auto-scaling.
W0518 13:19:20.485] #          See https://cloud.google.com/kubernetes-engine/docs/concepts/node-images#modifications
W0518 13:19:20.485] #          for more information.
W0518 13:19:20.485] ##############################################################################
W0518 13:19:21.046] scp: /var/log/fluentd.log*: No such file or directory
W0518 13:19:21.046] scp: /var/log/node-problem-detector.log*: No such file or directory
W0518 13:19:21.047] scp: /var/log/kubelet.cov*: No such file or directory
W0518 13:19:21.051] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1].
W0518 13:19:21.156] 2019/05/18 13:19:21 process.go:155: Step 'bash -c 
W0518 13:19:21.157] function log_dump_custom_get_instances() {
W0518 13:19:21.157]   if [[ $1 == "master" ]]; then
W0518 13:19:21.157]     return 0
W0518 13:19:21.157]   fi
W0518 13:19:21.157] 
... skipping 48 lines ...
W0518 13:23:57.687] Listed 0 items.
W0518 13:23:58.220] Listed 0 items.
W0518 13:23:58.275] 2019/05/18 13:23:58 process.go:155: Step './cluster/gce/list-resources.sh' finished in 9.253672657s
W0518 13:23:58.276] 2019/05/18 13:23:58 process.go:153: Running: diff -sw -U0 -F^\[.*\]$ /workspace/_artifacts/gcp-resources-before.txt /workspace/_artifacts/gcp-resources-after.txt
W0518 13:23:58.278] 2019/05/18 13:23:58 process.go:155: Step 'diff -sw -U0 -F^\[.*\]$ /workspace/_artifacts/gcp-resources-before.txt /workspace/_artifacts/gcp-resources-after.txt' finished in 1.454781ms
W0518 13:23:58.278] 2019/05/18 13:23:58 process.go:96: Saved XML output to /workspace/_artifacts/junit_runner.xml.
W0518 13:23:58.296] 2019/05/18 13:23:58 main.go:314: Something went wrong: encountered 1 errors: [error starting ./cluster/kubectl.sh --match-server-version=false version: exec: already started]
W0518 13:23:58.301] Traceback (most recent call last):
W0518 13:23:58.301]   File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 778, in <module>
W0518 13:23:58.301]     main(parse_args())
W0518 13:23:58.301]   File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 626, in main
W0518 13:23:58.301]     mode.start(runner_args)
W0518 13:23:58.302]   File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 262, in start
W0518 13:23:58.302]     check_env(env, self.command, *args)
W0518 13:23:58.302]   File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 111, in check_env
W0518 13:23:58.302]     subprocess.check_call(cmd, env=env)
W0518 13:23:58.302]   File "/usr/lib/python2.7/subprocess.py", line 186, in check_call
W0518 13:23:58.302]     raise CalledProcessError(retcode, cmd)
W0518 13:23:58.302] subprocess.CalledProcessError: Command '('kubetest', '--dump=/workspace/_artifacts', '--gcp-service-account=/etc/service-account/service-account.json', '--up', '--down', '--test', '--deployment=gke', '--provider=gke', '--cluster=test-0101dc0a7b', '--gcp-network=test-0101dc0a7b', '--check-leaked-resources', '--gcp-zone=us-west1-b', '--gcp-cloud-sdk=gs://cloud-sdk-testing/ci/staging', '--gke-environment=test', '--image-family=pipeline-2', '--image-project=ubuntu-os-gke-cloud-devel', '--gcp-node-image=custom', '--extract=ci/k8s-stable3', '--timeout=180m', '--test_args=--ginkgo.focus=\\[Feature:Reboot\\] --minStartupPods=8')' returned non-zero exit status 1
E0518 13:23:58.312] Command failed
I0518 13:23:58.313] process 259 exited with code 1 after 59.4m
E0518 13:23:58.313] FAIL: ci-kubernetes-e2e-gke-ubuntu2-k8sstable3-reboot
I0518 13:23:58.313] Call:  gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json
W0518 13:23:58.871] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com]
I0518 13:23:58.926] process 3093 exited with code 0 after 0.0m
I0518 13:23:58.926] Call:  gcloud config get-value account
I0518 13:23:59.263] process 3105 exited with code 0 after 0.0m
I0518 13:23:59.264] Will upload results to gs://kubernetes-jenkins/logs using pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com
I0518 13:23:59.264] Upload result and artifacts...
I0518 13:23:59.264] Gubernator results at https://gubernator.k8s.io/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-ubuntu2-k8sstable3-reboot/1129724284269236225
I0518 13:23:59.264] Call:  gsutil ls gs://kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-ubuntu2-k8sstable3-reboot/1129724284269236225/artifacts
W0518 13:24:00.427] CommandException: One or more URLs matched no objects.
E0518 13:24:00.568] Command failed
I0518 13:24:00.569] process 3117 exited with code 1 after 0.0m
W0518 13:24:00.569] Remote dir gs://kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-ubuntu2-k8sstable3-reboot/1129724284269236225/artifacts not exist yet
I0518 13:24:00.569] Call:  gsutil -m -q -o GSUtil:use_magicfile=True cp -r -c -z log,txt,xml /workspace/_artifacts gs://kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-ubuntu2-k8sstable3-reboot/1129724284269236225/artifacts
I0518 13:24:03.150] process 3259 exited with code 0 after 0.0m
I0518 13:24:03.151] Call:  git rev-parse HEAD
W0518 13:24:03.155] fatal: Not a git repository (or any of the parent directories): .git
E0518 13:24:03.156] Command failed
I0518 13:24:03.156] process 3897 exited with code 128 after 0.0m
I0518 13:24:03.156] Call:  git rev-parse HEAD
I0518 13:24:03.161] process 3898 exited with code 0 after 0.0m
I0518 13:24:03.161] Call:  gsutil stat gs://kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-ubuntu2-k8sstable3-reboot/jobResultsCache.json
I0518 13:24:04.406] process 3899 exited with code 0 after 0.0m
I0518 13:24:04.408] Call:  gsutil -q cat 'gs://kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-ubuntu2-k8sstable3-reboot/jobResultsCache.json#1558164282556978'
... skipping 8 lines ...