ci-kubernetes-e2e-gce-scale-performance #1347317422025084928
Recent runs || View in Spyglass
error during /go/src/k8s.io/perf-tests/run-e2e.sh cluster-loader2 --experimental-gcp-snapshot-prometheus-disk=true --experimental-prometheus-disk-snapshot-name=ci-kubernetes-e2e-gce-scale-performance-1347317422025084928 --nodes=5000 --provider=gce --report-dir=/workspace/_artifacts --testconfig=testing/load/config.yaml --testconfig=testing/access-tokens/config.yaml --testoverrides=./testing/experiments/enable_restart_count_check.yaml --testoverrides=./testing/experiments/ignore_known_gce_container_restarts.yaml --testoverrides=./testing/overrides/5000_nodes.yaml: exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=ClusterLoaderV2\sload\soverall\s\(testing\/load\/config\.yaml\)$'
:0
[measurement call APIResponsivenessPrometheus - APIResponsivenessPrometheusSimple error: top latency metric: there should be no high-latency requests, but: [got: &{Resource:pods Subresource: Verb:LIST Scope:cluster Latency:perc50: 1m0s, perc90: 1m0s, perc99: 1m0s Count:27 SlowCount:17}; expected perc99 <= 30s]]
:0
from junit.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=ClusterLoaderV2\sload\:\s\[step\:\s25\]\sCollecting\smeasurements$'
:0
[measurement call APIResponsivenessPrometheus - APIResponsivenessPrometheusSimple error: top latency metric: there should be no high-latency requests, but: [got: &{Resource:pods Subresource: Verb:LIST Scope:cluster Latency:perc50: 1m0s, perc90: 1m0s, perc99: 1m0s Count:27 SlowCount:17}; expected perc99 <= 30s]]
:0
from junit.xml
Filter through log files | View test history on testgrid
Check APIReachability
ClusterLoaderV2 access-tokens overall (testing/access-tokens/config.yaml)
ClusterLoaderV2 access-tokens: [step: 01] Starting measurements
ClusterLoaderV2 access-tokens: [step: 02] Creating ServiceAccounts
ClusterLoaderV2 access-tokens: [step: 03] Creating Tokens
ClusterLoaderV2 access-tokens: [step: 04] Starting measurement for waiting for pods
ClusterLoaderV2 access-tokens: [step: 05] Creating pods
ClusterLoaderV2 access-tokens: [step: 06] Waiting for pods to be running
ClusterLoaderV2 access-tokens: [step: 07] Wait 5min
ClusterLoaderV2 access-tokens: [step: 08] Deleting pods
ClusterLoaderV2 access-tokens: [step: 09] Waiting for pods to be deleted
ClusterLoaderV2 access-tokens: [step: 10] Collecting measurements
ClusterLoaderV2 load: [step: 01] Starting measurements
ClusterLoaderV2 load: [step: 02] Creating SVCs
ClusterLoaderV2 load: [step: 03] Creating PriorityClass for DaemonSets
ClusterLoaderV2 load: [step: 04] Starting measurement for waiting for pods
ClusterLoaderV2 load: [step: 05] Creating objects
ClusterLoaderV2 load: [step: 06] Waiting for pods to be running
ClusterLoaderV2 load: [step: 07] Creating scheduler throughput measurements
ClusterLoaderV2 load: [step: 08] Creating scheduler throughput pods
ClusterLoaderV2 load: [step: 09] Waiting for scheduler throughput pods to be created
ClusterLoaderV2 load: [step: 10] Collecting scheduler throughput measurements
ClusterLoaderV2 load: [step: 11] Deleting scheduler throughput pods
ClusterLoaderV2 load: [step: 12] Waiting for scheduler throughput pods to be deleted
ClusterLoaderV2 load: [step: 13] Starting latency pod measurements
ClusterLoaderV2 load: [step: 14] Creating latency pods
ClusterLoaderV2 load: [step: 15] Waiting for latency pods to be running
ClusterLoaderV2 load: [step: 16] Deleting latency pods
ClusterLoaderV2 load: [step: 17] Waiting for latency pods to be deleted
ClusterLoaderV2 load: [step: 18] Collecting pod startup latency
ClusterLoaderV2 load: [step: 19] Scaling and updating objects
ClusterLoaderV2 load: [step: 20] Waiting for objects to become scaled
ClusterLoaderV2 load: [step: 21] Deleting objects
ClusterLoaderV2 load: [step: 22] Waiting for pods to be deleted
ClusterLoaderV2 load: [step: 23] Deleting PriorityClass for DaemonSets
ClusterLoaderV2 load: [step: 24] Deleting SVCs
Deferred TearDown
DumpClusterLogs
Extract
TearDown
TearDown Previous
Timeout
Up
list nodes
test setup