periodic-cluster-api-provider-vsphere-e2e-main #1615035053480349696
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 0 failed / 0 succeeded |
| Started | |
| Elapsed | 1h5m |
| Revision | main |
... skipping 579 lines ... INFO: Waiting for rolling upgrade to start. INFO: Waiting for MachineDeployment rolling upgrade to start INFO: Waiting for rolling upgrade to complete. INFO: Waiting for MachineDeployment rolling upgrade to complete [1mSTEP:[0m PASSED! [38;5;243m@ 01/16/23 17:29:08.258[0m [1mSTEP:[0m Dumping logs from the "md-rollout-410eao" workload cluster [38;5;243m@ 01/16/23 17:29:08.258[0m Failed to get logs for Machine md-rollout-410eao-c9pqb, Cluster md-rollout-mk25ai/md-rollout-410eao: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 Failed to get logs for Machine md-rollout-410eao-md-0-668497c6dc-jnztl, Cluster md-rollout-mk25ai/md-rollout-410eao: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 [1mSTEP:[0m Dumping all the Cluster API resources in the "md-rollout-mk25ai" namespace [38;5;243m@ 01/16/23 17:29:12.821[0m [1mSTEP:[0m Deleting cluster md-rollout-mk25ai/md-rollout-410eao [38;5;243m@ 01/16/23 17:29:13.137[0m [1mSTEP:[0m Deleting cluster md-rollout-410eao [38;5;243m@ 01/16/23 17:29:13.156[0m INFO: Waiting for the Cluster md-rollout-mk25ai/md-rollout-410eao to be deleted [1mSTEP:[0m Waiting for cluster md-rollout-410eao to be deleted [38;5;243m@ 01/16/23 17:29:13.17[0m [1mSTEP:[0m Deleting namespace used for hosting the "md-rollout" test spec [38;5;243m@ 01/16/23 17:29:43.19[0m ... skipping 114 lines ... [1mSTEP:[0m Waiting for deployment node-drain-w8cgd4-unevictable-workload/unevictable-pod-dri to be available [38;5;243m@ 01/16/23 17:42:19.467[0m [1mSTEP:[0m Scale down the controlplane of the workload cluster and make sure that nodes running workload can be deleted even the draining process is blocked. [38;5;243m@ 01/16/23 17:42:29.773[0m INFO: Scaling controlplane node-drain-w8cgd4/node-drain-uipwh6 from 3 to 1 replicas INFO: Waiting for correct number of replicas to exist [1mSTEP:[0m PASSED! [38;5;243m@ 01/16/23 17:46:10.369[0m [1mSTEP:[0m Dumping logs from the "node-drain-uipwh6" workload cluster [38;5;243m@ 01/16/23 17:46:10.37[0m Failed to get logs for Machine node-drain-uipwh6-9lnxq, Cluster node-drain-w8cgd4/node-drain-uipwh6: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 [1mSTEP:[0m Dumping all the Cluster API resources in the "node-drain-w8cgd4" namespace [38;5;243m@ 01/16/23 17:46:12.59[0m [1mSTEP:[0m Deleting cluster node-drain-w8cgd4/node-drain-uipwh6 [38;5;243m@ 01/16/23 17:46:12.891[0m [1mSTEP:[0m Deleting cluster node-drain-uipwh6 [38;5;243m@ 01/16/23 17:46:12.913[0m INFO: Waiting for the Cluster node-drain-w8cgd4/node-drain-uipwh6 to be deleted [1mSTEP:[0m Waiting for cluster node-drain-uipwh6 to be deleted [38;5;243m@ 01/16/23 17:46:12.926[0m [1mSTEP:[0m Deleting namespace used for hosting the "node-drain" test spec [38;5;243m@ 01/16/23 17:46:42.947[0m ... skipping 108 lines ... INFO: Waiting for correct number of replicas to exist [1mSTEP:[0m Scaling the MachineDeployment down to 1 [38;5;243m@ 01/16/23 17:56:52.086[0m INFO: Scaling machine deployment md-scale-33aaxy/md-scale-emffik-md-0 from 3 to 1 replicas INFO: Waiting for correct number of replicas to exist [1mSTEP:[0m PASSED! [38;5;243m@ 01/16/23 17:57:02.202[0m [1mSTEP:[0m Dumping logs from the "md-scale-emffik" workload cluster [38;5;243m@ 01/16/23 17:57:02.202[0m Failed to get logs for Machine md-scale-emffik-hqjbf, Cluster md-scale-33aaxy/md-scale-emffik: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 Failed to get logs for Machine md-scale-emffik-md-0-85759bbf99-wfddp, Cluster md-scale-33aaxy/md-scale-emffik: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 [1mSTEP:[0m Dumping all the Cluster API resources in the "md-scale-33aaxy" namespace [38;5;243m@ 01/16/23 17:57:06.706[0m [1mSTEP:[0m Deleting cluster md-scale-33aaxy/md-scale-emffik [38;5;243m@ 01/16/23 17:57:07.041[0m [1mSTEP:[0m Deleting cluster md-scale-emffik [38;5;243m@ 01/16/23 17:57:07.062[0m INFO: Waiting for the Cluster md-scale-33aaxy/md-scale-emffik to be deleted [1mSTEP:[0m Waiting for cluster md-scale-emffik to be deleted [38;5;243m@ 01/16/23 17:57:07.078[0m [1mSTEP:[0m Deleting namespace used for hosting the "md-scale" test spec [38;5;243m@ 01/16/23 17:57:37.099[0m ... skipping 52 lines ... Patching MachineHealthCheck unhealthy condition to one of the nodes INFO: Patching the node condition to the node Waiting for remediation Waiting until the node with unhealthy node condition is remediated [1mSTEP:[0m PASSED! [38;5;243m@ 01/16/23 18:03:33.552[0m [1mSTEP:[0m Dumping logs from the "mhc-remediation-qm18x4" workload cluster [38;5;243m@ 01/16/23 18:03:33.553[0m Failed to get logs for Machine mhc-remediation-qm18x4-dr4dv, Cluster mhc-remediation-mwxyc6/mhc-remediation-qm18x4: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 Failed to get logs for Machine mhc-remediation-qm18x4-md-0-74567686f5-64xlq, Cluster mhc-remediation-mwxyc6/mhc-remediation-qm18x4: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 [1mSTEP:[0m Dumping all the Cluster API resources in the "mhc-remediation-mwxyc6" namespace [38;5;243m@ 01/16/23 18:03:37.747[0m [1mSTEP:[0m Deleting cluster mhc-remediation-mwxyc6/mhc-remediation-qm18x4 [38;5;243m@ 01/16/23 18:03:38.047[0m [1mSTEP:[0m Deleting cluster mhc-remediation-qm18x4 [38;5;243m@ 01/16/23 18:03:38.068[0m INFO: Waiting for the Cluster mhc-remediation-mwxyc6/mhc-remediation-qm18x4 to be deleted [1mSTEP:[0m Waiting for cluster mhc-remediation-qm18x4 to be deleted [38;5;243m@ 01/16/23 18:03:38.082[0m [1mSTEP:[0m Deleting namespace used for hosting the "mhc-remediation" test spec [38;5;243m@ 01/16/23 18:04:08.102[0m ... skipping 54 lines ... Patching MachineHealthCheck unhealthy condition to one of the nodes INFO: Patching the node condition to the node Waiting for remediation Waiting until the node with unhealthy node condition is remediated [1mSTEP:[0m PASSED! [38;5;243m@ 01/16/23 18:12:47.687[0m [1mSTEP:[0m Dumping logs from the "mhc-remediation-wsgqej" workload cluster [38;5;243m@ 01/16/23 18:12:47.688[0m Failed to get logs for Machine mhc-remediation-wsgqej-b28hs, Cluster mhc-remediation-d39e8w/mhc-remediation-wsgqej: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 Failed to get logs for Machine mhc-remediation-wsgqej-bnzkn, Cluster mhc-remediation-d39e8w/mhc-remediation-wsgqej: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 Failed to get logs for Machine mhc-remediation-wsgqej-md-0-594d5c8b8-2fht9, Cluster mhc-remediation-d39e8w/mhc-remediation-wsgqej: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 Failed to get logs for Machine mhc-remediation-wsgqej-q2kjv, Cluster mhc-remediation-d39e8w/mhc-remediation-wsgqej: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 [1mSTEP:[0m Dumping all the Cluster API resources in the "mhc-remediation-d39e8w" namespace [38;5;243m@ 01/16/23 18:12:55.73[0m [1mSTEP:[0m Deleting cluster mhc-remediation-d39e8w/mhc-remediation-wsgqej [38;5;243m@ 01/16/23 18:12:56.058[0m [1mSTEP:[0m Deleting cluster mhc-remediation-wsgqej [38;5;243m@ 01/16/23 18:12:56.079[0m INFO: Waiting for the Cluster mhc-remediation-d39e8w/mhc-remediation-wsgqej to be deleted [1mSTEP:[0m Waiting for cluster mhc-remediation-wsgqej to be deleted [38;5;243m@ 01/16/23 18:12:56.092[0m [1mSTEP:[0m Deleting namespace used for hosting the "mhc-remediation" test spec [38;5;243m@ 01/16/23 18:13:46.125[0m ... skipping 123 lines ... | for { [1m[38;5;214m> select {[0m | case <-ctx.Done(): | return [38;5;214m[1m> sigs.k8s.io/cluster-api/test/framework.WatchPodMetrics[0m [38;5;214m[1m/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:225[0m | }, retryableOperationTimeout, retryableOperationInterval).Should(Succeed(), "Failed to list Pods for deployment %s", klog.KObj(input.Deployment)) | [1m[38;5;214m> go func() {[0m | defer GinkgoRecover() | for { [38;5;214mgoroutine 31836 [sync.Cond.Wait, 6 minutes][0m ... skipping 18 lines ... [38;5;214m[1m> sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs.func2({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc000c2c3c0, 0x3a}, {0xc000c2c400, 0x35}, {0xc0009a4d20, 0x1d}, ...}, ...}, ...)[0m [38;5;214m[1m/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:186[0m | out := bufio.NewWriter(f) | defer out.Flush() [1m[38;5;214m> _, err = out.ReadFrom(podLogs)[0m | if err != nil && err != io.ErrUnexpectedEOF { | // Failing to stream logs should not cause the test to fail [38;5;214m[1m> sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs[0m [38;5;214m[1m/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:161[0m | | // Watch each container's logs in a goroutine so we can stream them all concurrently. [1m[38;5;214m> go func(pod corev1.Pod, container corev1.Container) {[0m | defer GinkgoRecover() ... skipping 21 lines ... [38;5;214m[1m> sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs.func2({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc000c2d440, 0x3e}, {0xc000c2d480, 0x39}, {0xc0023a8270, 0x21}, ...}, ...}, ...)[0m [38;5;214m[1m/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:186[0m | out := bufio.NewWriter(f) | defer out.Flush() [1m[38;5;214m> _, err = out.ReadFrom(podLogs)[0m | if err != nil && err != io.ErrUnexpectedEOF { | // Failing to stream logs should not cause the test to fail [38;5;214m[1m> sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs[0m [38;5;214m[1m/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:161[0m | | // Watch each container's logs in a goroutine so we can stream them all concurrently. [1m[38;5;214m> go func(pod corev1.Pod, container corev1.Container) {[0m | defer GinkgoRecover() ... skipping 6 lines ... | for { [1m[38;5;214m> select {[0m | case <-ctx.Done(): | return [38;5;214m[1m> sigs.k8s.io/cluster-api/test/framework.WatchPodMetrics[0m [38;5;214m[1m/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:225[0m | }, retryableOperationTimeout, retryableOperationInterval).Should(Succeed(), "Failed to list Pods for deployment %s", klog.KObj(input.Deployment)) | [1m[38;5;214m> go func() {[0m | defer GinkgoRecover() | for { [38;5;214mgoroutine 31712 [sync.Cond.Wait][0m ... skipping 18 lines ... [38;5;214m[1m> sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs.func2({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc001f68fc0, 0x28}, {0xc001f68ff0, 0x23}, {0xc000fa0da0, 0xb}, ...}, ...}, ...)[0m [38;5;214m[1m/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:186[0m | out := bufio.NewWriter(f) | defer out.Flush() [1m[38;5;214m> _, err = out.ReadFrom(podLogs)[0m | if err != nil && err != io.ErrUnexpectedEOF { | // Failing to stream logs should not cause the test to fail [38;5;214m[1m> sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs[0m [38;5;214m[1m/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:161[0m | | // Watch each container's logs in a goroutine so we can stream them all concurrently. [1m[38;5;214m> go func(pod corev1.Pod, container corev1.Container) {[0m | defer GinkgoRecover() ... skipping 6 lines ... | for { [1m[38;5;214m> select {[0m | case <-ctx.Done(): | return [38;5;214m[1m> sigs.k8s.io/cluster-api/test/framework.WatchPodMetrics[0m [38;5;214m[1m/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:225[0m | }, retryableOperationTimeout, retryableOperationInterval).Should(Succeed(), "Failed to list Pods for deployment %s", klog.KObj(input.Deployment)) | [1m[38;5;214m> go func() {[0m | defer GinkgoRecover() | for { [38;5;214mgoroutine 73065 [chan receive, 6 minutes][0m ... skipping 41 lines ... [38;5;214m[1m> sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs.func2({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc0024167e0, 0x27}, {0xc002416810, 0x22}, {0xc002530c70, 0xb}, ...}, ...}, ...)[0m [38;5;214m[1m/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:186[0m | out := bufio.NewWriter(f) | defer out.Flush() [1m[38;5;214m> _, err = out.ReadFrom(podLogs)[0m | if err != nil && err != io.ErrUnexpectedEOF { | // Failing to stream logs should not cause the test to fail [38;5;214m[1m> sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs[0m [38;5;214m[1m/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:161[0m | | // Watch each container's logs in a goroutine so we can stream them all concurrently. [1m[38;5;214m> go func(pod corev1.Pod, container corev1.Container) {[0m | defer GinkgoRecover() ... skipping 6 lines ... | for { [1m[38;5;214m> select {[0m | case <-ctx.Done(): | return [38;5;214m[1m> sigs.k8s.io/cluster-api/test/framework.WatchPodMetrics[0m [38;5;214m[1m/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:225[0m | }, retryableOperationTimeout, retryableOperationInterval).Should(Succeed(), "Failed to list Pods for deployment %s", klog.KObj(input.Deployment)) | [1m[38;5;214m> go func() {[0m | defer GinkgoRecover() | for { [38;5;243m------------------------------[0m [38;5;14mS [SKIPPED][0m ... skipping 29 lines ... [38;5;243m/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/e2e/clusterctl_upgrade.go:191[0m [38;5;243m------------------------------[0m [0m[1m[SynchronizedAfterSuite] [0m [38;5;243m/home/prow/go/src/sigs.k8s.io/cluster-api-provider-vsphere/test/e2e/e2e_suite_test.go:159[0m [1mSTEP:[0m Cleaning up the vSphere session [38;5;243m@ 01/16/23 18:20:26.519[0m [1mSTEP:[0m Tearing down the management cluster [38;5;243m@ 01/16/23 18:20:26.693[0m INFO: Failed to list the machines: Get "https://127.0.0.1:42883/apis/cluster.x-k8s.io/v1beta1/namespaces/capv-e2e-8tka9a/machines?labelSelector=cluster.x-k8s.io%2Fcluster-name%3Dstorage-policy-cdbx87%2Ccluster.x-k8s.io%2Fcontrol-plane%3D": dial tcp 127.0.0.1:42883: connect: connection refused [38;5;10m[SynchronizedAfterSuite] PASSED [1.488 seconds][0m [38;5;243m------------------------------[0m [38;5;9m[1mSummarizing 1 Failure:[0m [38;5;214m[TIMEDOUT][0m [0mCluster creation with storage policy [38;5;214m[1m[It] should create a cluster successfully[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/cluster-api-provider-vsphere/test/e2e/storage_policy_test.go:57[0m [38;5;9m[1mRan 8 of 17 Specs in 3567.645 seconds[0m [38;5;9m[1mFAIL! - Suite Timeout Elapsed[0m -- [38;5;10m[1m7 Passed[0m | [38;5;9m[1m1 Failed[0m | [38;5;11m[1m1 Pending[0m | [38;5;14m[1m8 Skipped[0m --- FAIL: TestE2E (3567.65s) FAIL Ginkgo ran 1 suite in 1h0m21.947154696s Test Suite Failed real 60m21.968s user 5m47.570s sys 1m8.697s make: *** [Makefile:183: e2e] Error 1 Releasing IP claims ipclaim.ipam.metal3.io "ip-claim-e503d19725d61c58c5a22018958b64f93644799e" deleted ipclaim.ipam.metal3.io "workload-ip-claim-8af8800409345f027ac73aff802318884a72eb43" deleted vpn WARNING: [capv-prow@cluster-api-provider-vsphere.iam.gserviceaccount.com] appears to be a service account. Service account tokens cannot be revoked, but they will expire automatically. To prevent use of the service account token earlier than the expiration, delete or disable the parent service account. To explicitly delete the key associated with the service account use `gcloud iam service-accounts keys delete` instead`. Revoked credentials: ... skipping 13 lines ...