This job view page is being replaced by Spyglass soon. Check out the new job view.
PRRainbowMango: Migrate prometheus bucket functionality to kube-metrics for winkernel
ResultFAILURE
Tests 1 failed / 7 succeeded
Started2019-09-16 03:49
Elapsed30m51s
Revision
Buildergke-prow-ssd-pool-1a225945-810g
Refs master:ba075272
82737:d36c27a1
podf005a504-d834-11e9-857c-ce22559442cb
infra-commite1cbc3ccd
job-versionv1.17.0-alpha.0.1443+e25b59fc964011
podf005a504-d834-11e9-857c-ce22559442cb
repok8s.io/kubernetes
repo-commite25b59fc964011bdd7eb36fb20c76f9d22b18e21
repos{u'k8s.io/kubernetes': u'master:ba07527278ef2cde9c27886ec3333cfef472112a,82737:d36c27a1068e172c7d87d7b9a0a6c25f2e313bdd', u'k8s.io/perf-tests': u'master', u'k8s.io/release': u'master'}
revisionv1.17.0-alpha.0.1443+e25b59fc964011

Test Failures


Up 7m5s

error during ./hack/e2e-internal/e2e-up.sh: exit status 2
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 7 Passed Tests

Error lines from build-log.txt

... skipping 961 lines ...
W0916 04:04:56.153] Zone: us-east1-b
I0916 04:04:59.418] Bringing down cluster
W0916 04:04:59.519] INSTANCE_GROUPS=e2e-82737-ac87c-minion-group
W0916 04:04:59.519] NODE_NAMES=e2e-82737-ac87c-minion-group-9xt3
W0916 04:05:02.218] Deleting Managed Instance Group...
W0916 04:05:02.608] done.
W0916 04:05:02.611] ERROR: (gcloud.compute.instance-groups.managed.delete) Some requests did not succeed:
W0916 04:05:02.612]  - The resource 'projects/k8s-presubmit-scale/zones/us-east1-b/instanceGroupManagers/e2e-82737-ac87c-minion-group' is not ready
W0916 04:05:02.612] 
W0916 04:05:02.690] Failed to delete instance group(s).
W0916 04:05:04.991] ERROR: (gcloud.compute.instance-templates.delete) Could not fetch resource:
W0916 04:05:04.992]  - The instance_template resource 'projects/k8s-presubmit-scale/global/instanceTemplates/e2e-82737-ac87c-minion-template' is already being used by 'projects/k8s-presubmit-scale/zones/us-east1-b/instanceGroupManagers/e2e-82737-ac87c-minion-group'
W0916 04:05:04.992] 
W0916 04:05:15.009] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-presubmit-scale/global/instanceTemplates/e2e-82737-ac87c-windows-node-template].
W0916 04:05:21.933] Warning: Permanently added 'compute.5210533250590750830' (ED25519) to the list of known hosts.
I0916 04:05:22.399] Removing etcd replica, name: e2e-82737-ac87c-master, port: 2379, result: 52
I0916 04:05:24.136] {"message":"Internal Server Error"}Removing etcd replica, name: e2e-82737-ac87c-master, port: 4002, result: 0
W0916 04:05:31.125] Updated [https://www.googleapis.com/compute/v1/projects/k8s-presubmit-scale/zones/us-east1-b/instances/e2e-82737-ac87c-master].
W0916 04:08:06.966] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-presubmit-scale/zones/us-east1-b/instances/e2e-82737-ac87c-master].
W0916 04:08:11.716] ERROR: (gcloud.compute.firewall-rules.delete) Could not fetch resource:
W0916 04:08:11.717]  - The resource 'projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-master-https' is not ready
W0916 04:08:11.717] 
W0916 04:08:12.462] ERROR: (gcloud.compute.firewall-rules.delete) Could not fetch resource:
W0916 04:08:12.463]  - The resource 'projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-master-etcd' is not ready
W0916 04:08:12.463] 
W0916 04:08:13.260] ERROR: (gcloud.compute.firewall-rules.delete) Could not fetch resource:
W0916 04:08:13.260]  - The resource 'projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-minion-all' is not ready
W0916 04:08:13.260] 
W0916 04:08:13.353] Failed to delete firewall rules.
W0916 04:08:22.148] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-presubmit-scale/regions/us-east1/addresses/e2e-82737-ac87c-master-ip].
W0916 04:08:49.821] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-default-internal-master].
W0916 04:08:50.656] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-default-internal-node].
W0916 04:08:51.279] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-default-ssh].
I0916 04:08:52.676] Deleting firewall rules remaining in network e2e-82737-ac87c: e2e-82737-ac87c-kubemark-default-internal-master
I0916 04:08:52.676] e2e-82737-ac87c-kubemark-default-internal-node
I0916 04:08:52.677] e2e-82737-ac87c-kubemark-master-etcd
I0916 04:08:52.677] e2e-82737-ac87c-kubemark-master-https
I0916 04:08:52.677] e2e-82737-ac87c-kubemark-minion-all
I0916 04:08:52.678] e2e-82737-ac87c-kubemark-minion-http-alt
I0916 04:08:52.678] e2e-82737-ac87c-kubemark-minion-nodeports
W0916 04:08:54.806] ERROR: (gcloud.compute.firewall-rules.delete) Could not fetch resource:
W0916 04:08:54.807]  - The resource 'projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-kubemark-default-internal-master' is not ready
W0916 04:08:54.807] 
W0916 04:08:55.800] ERROR: (gcloud.compute.firewall-rules.delete) Could not fetch resource:
W0916 04:08:55.800]  - The resource 'projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-kubemark-default-internal-node' is not ready
W0916 04:08:55.800] 
W0916 04:08:56.622] ERROR: (gcloud.compute.firewall-rules.delete) Could not fetch resource:
W0916 04:08:56.622]  - The resource 'projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-kubemark-master-etcd' is not ready
W0916 04:08:56.622] 
W0916 04:08:57.402] ERROR: (gcloud.compute.firewall-rules.delete) Could not fetch resource:
W0916 04:08:57.403]  - The resource 'projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-kubemark-master-https' is not ready
W0916 04:08:57.403] 
W0916 04:08:58.287] ERROR: (gcloud.compute.firewall-rules.delete) Could not fetch resource:
W0916 04:08:58.287]  - The resource 'projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-kubemark-minion-all' is not ready
W0916 04:08:58.287] 
W0916 04:08:59.306] ERROR: (gcloud.compute.firewall-rules.delete) Could not fetch resource:
W0916 04:08:59.307]  - The resource 'projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-kubemark-minion-http-alt' is not ready
W0916 04:08:59.307] 
W0916 04:08:59.939] ERROR: (gcloud.compute.firewall-rules.delete) Could not fetch resource:
W0916 04:08:59.939]  - The resource 'projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-kubemark-minion-nodeports' is not ready
W0916 04:08:59.940] 
W0916 04:09:00.027] Failed to delete firewall rules.
I0916 04:09:00.925] Deleting custom subnet...
W0916 04:09:01.907] ERROR: (gcloud.compute.networks.subnets.delete) Could not fetch resource:
W0916 04:09:01.908]  - The subnetwork resource 'projects/k8s-presubmit-scale/regions/us-east1/subnetworks/e2e-82737-ac87c-custom-subnet' is already being used by 'projects/k8s-presubmit-scale/zones/us-east1-b/instances/e2e-82737-ac87c-kubemark-master'
W0916 04:09:01.908] 
W0916 04:09:07.371] ERROR: (gcloud.compute.networks.delete) Could not fetch resource:
W0916 04:09:07.372]  - The network resource 'projects/k8s-presubmit-scale/global/networks/e2e-82737-ac87c' is already being used by 'projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-kubemark-minion-nodeports'
W0916 04:09:07.372] 
I0916 04:09:07.472] Failed to delete network 'e2e-82737-ac87c'. Listing firewall-rules:
W0916 04:09:08.294] 
W0916 04:09:08.294] To show all fields of the firewall, please show in JSON format: --format=json
W0916 04:09:08.294] To show all fields in table format, please see the examples in --help.
W0916 04:09:08.294] 
W0916 04:09:08.556] W0916 04:09:08.555977   79733 loader.go:223] Config not found: /workspace/.kube/config
W0916 04:09:08.731] W0916 04:09:08.730834   79779 loader.go:223] Config not found: /workspace/.kube/config
... skipping 131 lines ...
W0916 04:11:05.330] NODE_NAMES=e2e-82737-ac87c-minion-group-3qh6 e2e-82737-ac87c-minion-group-7kc9 e2e-82737-ac87c-minion-group-k7v3 e2e-82737-ac87c-minion-group-ljcf e2e-82737-ac87c-minion-group-r6xb e2e-82737-ac87c-minion-group-v23z e2e-82737-ac87c-minion-group-x5tj
W0916 04:11:05.330] Trying to find master named 'e2e-82737-ac87c-master'
W0916 04:11:05.331] Looking for address 'e2e-82737-ac87c-master-ip'
I0916 04:11:06.187] Waiting up to 300 seconds for cluster initialization.
I0916 04:11:06.188] 
I0916 04:11:06.188]   This will continually check to see if the API for kubernetes is reachable.
I0916 04:11:06.188]   This may time out if there was some uncaught error during start up.
I0916 04:11:06.188] 
W0916 04:11:06.289] Using master: e2e-82737-ac87c-master (external IP: 104.196.179.169; internal IP: (not set))
I0916 04:16:15.218] ...........................................Checking for custom logdump instances, if any
I0916 04:16:15.228] Sourcing kube-util.sh
I0916 04:16:15.306] Detecting project
I0916 04:16:15.326] Project: k8s-presubmit-scale
I0916 04:16:15.327] Network Project: k8s-presubmit-scale
I0916 04:16:15.327] Zone: us-east1-b
I0916 04:16:15.327] Dumping logs from master locally to '/workspace/_artifacts'
W0916 04:16:15.526] Cluster failed to initialize within 300 seconds.
W0916 04:16:15.653] Last output from querying API server follows:
W0916 04:16:15.780] -----------------------------------------------------
W0916 04:16:15.781]   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
W0916 04:16:15.782]                                  Dload  Upload   Total   Spent    Left  Speed
W0916 04:16:15.783] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:03 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:04 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:05 --:--:--     0
W0916 04:16:15.784] curl: (28) Connection timed out after 5029 milliseconds
... skipping 17 lines ...
W0916 04:16:58.657] scp: /var/log/glbc.log*: No such file or directory
W0916 04:16:58.657] scp: /var/log/cluster-autoscaler.log*: No such file or directory
W0916 04:16:58.658] scp: /var/log/kube-addon-manager.log*: No such file or directory
W0916 04:16:58.658] scp: /var/log/fluentd.log*: No such file or directory
W0916 04:16:58.658] scp: /var/log/kubelet.cov*: No such file or directory
W0916 04:16:58.658] scp: /var/log/startupscript.log*: No such file or directory
W0916 04:16:58.665] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1].
I0916 04:16:58.765] Dumping logs from nodes locally to '/workspace/_artifacts'
I0916 04:16:58.766] Detecting nodes in the cluster
I0916 04:17:01.288] No nodes found!
W0916 04:17:04.094] INSTANCE_GROUPS=
W0916 04:17:04.094] NODE_NAMES=
W0916 04:17:04.095] 2019/09/16 04:17:04 process.go:155: Step './cluster/log-dump/log-dump.sh /workspace/_artifacts' finished in 48.882032464s
... skipping 24 lines ...
W0916 04:19:38.394] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-presubmit-scale/regions/us-east1/addresses/e2e-82737-ac87c-master-ip].
W0916 04:20:01.996] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-default-internal-master].
W0916 04:20:08.010] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-default-internal-node].
W0916 04:20:09.343] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-presubmit-scale/global/firewalls/e2e-82737-ac87c-default-ssh].
I0916 04:20:10.332] Deleting firewall rules remaining in network e2e-82737-ac87c: 
I0916 04:20:11.218] Deleting custom subnet...
W0916 04:20:12.239] ERROR: (gcloud.compute.networks.subnets.delete) Could not fetch resource:
W0916 04:20:12.239]  - The subnetwork resource 'projects/k8s-presubmit-scale/regions/us-east1/subnetworks/e2e-82737-ac87c-custom-subnet' is already being used by 'projects/k8s-presubmit-scale/zones/us-east1-b/instances/e2e-82737-ac87c-kubemark-master'
W0916 04:20:12.240] 
W0916 04:20:20.978] ERROR: (gcloud.compute.networks.delete) Could not fetch resource:
W0916 04:20:20.978]  - The network resource 'projects/k8s-presubmit-scale/global/networks/e2e-82737-ac87c' is already being used by 'projects/k8s-presubmit-scale/zones/us-east1-b/instances/e2e-82737-ac87c-kubemark-master'
W0916 04:20:20.979] 
I0916 04:20:21.080] Failed to delete network 'e2e-82737-ac87c'. Listing firewall-rules:
W0916 04:20:22.151] 
W0916 04:20:22.151] To show all fields of the firewall, please show in JSON format: --format=json
W0916 04:20:22.152] To show all fields in table format, please see the examples in --help.
W0916 04:20:22.152] 
W0916 04:20:22.424] W0916 04:20:22.423861   85149 loader.go:223] Config not found: /workspace/.kube/config
W0916 04:20:22.625] W0916 04:20:22.624611   85195 loader.go:223] Config not found: /workspace/.kube/config
... skipping 6 lines ...
W0916 04:20:23.110] W0916 04:20:23.110432   85284 loader.go:223] Config not found: /workspace/.kube/config
I0916 04:20:23.211] Property "users.k8s-presubmit-scale_e2e-82737-ac87c-basic-auth" unset.
W0916 04:20:23.324] W0916 04:20:23.324013   85330 loader.go:223] Config not found: /workspace/.kube/config
W0916 04:20:23.324] W0916 04:20:23.324263   85330 loader.go:223] Config not found: /workspace/.kube/config
W0916 04:20:23.330] 2019/09/16 04:20:23 process.go:155: Step './hack/e2e-internal/e2e-down.sh' finished in 3m19.235571738s
W0916 04:20:23.346] 2019/09/16 04:20:23 process.go:96: Saved XML output to /workspace/_artifacts/junit_runner.xml.
W0916 04:20:23.357] 2019/09/16 04:20:23 main.go:319: Something went wrong: starting e2e cluster: error during ./hack/e2e-internal/e2e-up.sh: exit status 2
W0916 04:20:23.358] Traceback (most recent call last):
W0916 04:20:23.358]   File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 778, in <module>
W0916 04:20:23.368]     main(parse_args())
W0916 04:20:23.368]   File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 626, in main
W0916 04:20:23.368]     mode.start(runner_args)
W0916 04:20:23.368]   File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 262, in start
... skipping 3 lines ...
W0916 04:20:23.369]   File "/usr/lib/python2.7/subprocess.py", line 186, in check_call
W0916 04:20:23.373]     raise CalledProcessError(retcode, cmd)
W0916 04:20:23.375] subprocess.CalledProcessError: Command '('kubetest', '--dump=/workspace/_artifacts', '--gcp-service-account=/etc/service-account/service-account.json', '--build=bazel', '--stage=gs://kubernetes-release-pull/ci/pull-kubernetes-kubemark-e2e-gce-big', '--up', '--down', '--provider=gce', '--cluster=e2e-82737-ac87c', '--gcp-network=e2e-82737-ac87c', '--extract=local', '--gcp-master-size=n1-standard-4', '--gcp-node-size=n1-standard-8', '--gcp-nodes=7', '--gcp-project=k8s-presubmit-scale', '--gcp-zone=us-east1-b', '--kubemark', '--kubemark-nodes=500', '--test_args=--ginkgo.focus=xxxx', '--test-cmd=/go/src/k8s.io/perf-tests/run-e2e.sh', '--test-cmd-args=cluster-loader2', '--test-cmd-args=--nodes=500', '--test-cmd-args=--provider=kubemark', '--test-cmd-args=--report-dir=/workspace/_artifacts', '--test-cmd-args=--testconfig=testing/density/config.yaml', '--test-cmd-args=--testconfig=testing/load/config.yaml', '--test-cmd-args=--testoverrides=./testing/load/kubemark/500_nodes/override.yaml', '--test-cmd-args=--testoverrides=./testing/load/experimental/overrides/enable_configmaps.yaml', '--test-cmd-args=--testoverrides=./testing/load/experimental/overrides/enable_daemonsets.yaml', '--test-cmd-args=--testoverrides=./testing/load/experimental/overrides/enable_jobs.yaml', '--test-cmd-args=--testoverrides=./testing/load/experimental/overrides/enable_pvs.yaml', '--test-cmd-args=--testoverrides=./testing/load/experimental/overrides/enable_secrets.yaml', '--test-cmd-args=--testoverrides=./testing/load/experimental/overrides/enable_statefulsets.yaml', '--test-cmd-name=ClusterLoaderV2', '--timeout=100m')' returned non-zero exit status 1
I0916 04:20:23.391] Property "contexts.k8s-presubmit-scale_e2e-82737-ac87c" unset.
I0916 04:20:23.392] Cleared config for k8s-presubmit-scale_e2e-82737-ac87c from /workspace/.kube/config
I0916 04:20:23.392] Done
E0916 04:20:23.392] Command failed
I0916 04:20:23.392] process 700 exited with code 1 after 29.4m
E0916 04:20:23.393] FAIL: pull-kubernetes-kubemark-e2e-gce-big
I0916 04:20:23.393] Call:  gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json
W0916 04:20:24.172] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com]
I0916 04:20:24.247] process 85339 exited with code 0 after 0.0m
I0916 04:20:24.247] Call:  gcloud config get-value account
I0916 04:20:24.652] process 85351 exited with code 0 after 0.0m
I0916 04:20:24.652] Will upload results to gs://kubernetes-jenkins/pr-logs using pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com
I0916 04:20:24.653] Upload result and artifacts...
I0916 04:20:24.653] Gubernator results at https://gubernator.k8s.io/build/kubernetes-jenkins/pr-logs/pull/82737/pull-kubernetes-kubemark-e2e-gce-big/1173443769769398272
I0916 04:20:24.653] Call:  gsutil ls gs://kubernetes-jenkins/pr-logs/pull/82737/pull-kubernetes-kubemark-e2e-gce-big/1173443769769398272/artifacts
W0916 04:20:26.007] CommandException: One or more URLs matched no objects.
E0916 04:20:26.184] Command failed
I0916 04:20:26.185] process 85363 exited with code 1 after 0.0m
W0916 04:20:26.185] Remote dir gs://kubernetes-jenkins/pr-logs/pull/82737/pull-kubernetes-kubemark-e2e-gce-big/1173443769769398272/artifacts not exist yet
I0916 04:20:26.186] Call:  gsutil -m -q -o GSUtil:use_magicfile=True cp -r -c -z log,txt,xml /workspace/_artifacts gs://kubernetes-jenkins/pr-logs/pull/82737/pull-kubernetes-kubemark-e2e-gce-big/1173443769769398272/artifacts
I0916 04:20:28.325] process 85505 exited with code 0 after 0.0m
I0916 04:20:28.327] Call:  git rev-parse HEAD
I0916 04:20:28.332] process 86056 exited with code 0 after 0.0m
... skipping 20 lines ...