This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 0 failed / 0 succeeded
Started2023-01-17 17:16
Elapsed1h5m
Revisionmain

No Test Failures!

Error lines from build-log.txt

... skipping 645 lines ...
  INFO: Waiting for correct number of replicas to exist
  STEP: Scaling the MachineDeployment down to 1 @ 01/17/23 17:36:28.302
  INFO: Scaling machine deployment md-scale-g5rdf4/md-scale-0wzhw0-md-0 from 3 to 1 replicas
  INFO: Waiting for correct number of replicas to exist
  STEP: PASSED! @ 01/17/23 17:36:38.438
  STEP: Dumping logs from the "md-scale-0wzhw0" workload cluster @ 01/17/23 17:36:38.438
Failed to get logs for Machine md-scale-0wzhw0-md-0-54cd5b457-tkt97, Cluster md-scale-g5rdf4/md-scale-0wzhw0: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
Failed to get logs for Machine md-scale-0wzhw0-spmxb, Cluster md-scale-g5rdf4/md-scale-0wzhw0: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
  STEP: Dumping all the Cluster API resources in the "md-scale-g5rdf4" namespace @ 01/17/23 17:36:43.002
  STEP: Deleting cluster md-scale-g5rdf4/md-scale-0wzhw0 @ 01/17/23 17:36:43.333
  STEP: Deleting cluster md-scale-0wzhw0 @ 01/17/23 17:36:43.353
  INFO: Waiting for the Cluster md-scale-g5rdf4/md-scale-0wzhw0 to be deleted
  STEP: Waiting for cluster md-scale-0wzhw0 to be deleted @ 01/17/23 17:36:43.369
  STEP: Deleting namespace used for hosting the "md-scale" test spec @ 01/17/23 17:37:13.391
... skipping 44 lines ...
  INFO: Waiting for the machine deployments to be provisioned
  STEP: Waiting for the workload nodes to exist @ 01/17/23 17:39:34.982
  STEP: Checking all the machines controlled by quick-start-xzuyoo-md-0 are in the "<None>" failure domain @ 01/17/23 17:40:45.119
  INFO: Waiting for the machine pools to be provisioned
  STEP: PASSED! @ 01/17/23 17:40:45.164
  STEP: Dumping logs from the "quick-start-xzuyoo" workload cluster @ 01/17/23 17:40:45.164
Failed to get logs for Machine quick-start-xzuyoo-md-0-65f86bf76c-m6xf7, Cluster quick-start-tf6ntp/quick-start-xzuyoo: dialing host IP address at 192.168.6.133: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Failed to get logs for Machine quick-start-xzuyoo-zf4xn, Cluster quick-start-tf6ntp/quick-start-xzuyoo: dialing host IP address at 192.168.6.31: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
  STEP: Dumping all the Cluster API resources in the "quick-start-tf6ntp" namespace @ 01/17/23 17:40:47.636
  STEP: Deleting cluster quick-start-tf6ntp/quick-start-xzuyoo @ 01/17/23 17:40:47.956
  STEP: Deleting cluster quick-start-xzuyoo @ 01/17/23 17:40:47.978
  INFO: Waiting for the Cluster quick-start-tf6ntp/quick-start-xzuyoo to be deleted
  STEP: Waiting for cluster quick-start-xzuyoo to be deleted @ 01/17/23 17:40:47.994
  STEP: Deleting namespace used for hosting the "quick-start" test spec @ 01/17/23 17:41:18.016
... skipping 44 lines ...
  INFO: Waiting for the machine deployments to be provisioned
  STEP: Waiting for the workload nodes to exist @ 01/17/23 17:43:59.754
  STEP: Checking all the machines controlled by quick-start-yakrou-md-0-pmrf6 are in the "<None>" failure domain @ 01/17/23 17:45:19.866
  INFO: Waiting for the machine pools to be provisioned
  STEP: PASSED! @ 01/17/23 17:45:19.905
  STEP: Dumping logs from the "quick-start-yakrou" workload cluster @ 01/17/23 17:45:19.905
Failed to get logs for Machine quick-start-yakrou-58v8r-km6hq, Cluster quick-start-usxan2/quick-start-yakrou: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
Failed to get logs for Machine quick-start-yakrou-md-0-pmrf6-65cdbc84b9-dbpxl, Cluster quick-start-usxan2/quick-start-yakrou: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
  STEP: Dumping all the Cluster API resources in the "quick-start-usxan2" namespace @ 01/17/23 17:45:24.41
  STEP: Deleting cluster quick-start-usxan2/quick-start-yakrou @ 01/17/23 17:45:24.746
  STEP: Deleting cluster quick-start-yakrou @ 01/17/23 17:45:24.768
  INFO: Waiting for the Cluster quick-start-usxan2/quick-start-yakrou to be deleted
  STEP: Waiting for cluster quick-start-yakrou to be deleted @ 01/17/23 17:45:24.781
  STEP: Deleting namespace used for hosting the "quick-start" test spec @ 01/17/23 17:45:54.804
... skipping 174 lines ...
  INFO: Waiting for rolling upgrade to start.
  INFO: Waiting for MachineDeployment rolling upgrade to start
  INFO: Waiting for rolling upgrade to complete.
  INFO: Waiting for MachineDeployment rolling upgrade to complete
  STEP: PASSED! @ 01/17/23 18:03:34.294
  STEP: Dumping logs from the "md-rollout-8d7147" workload cluster @ 01/17/23 18:03:34.294
Failed to get logs for Machine md-rollout-8d7147-8xgt9, Cluster md-rollout-g7to6q/md-rollout-8d7147: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
Failed to get logs for Machine md-rollout-8d7147-md-0-589cd4d85b-2ht69, Cluster md-rollout-g7to6q/md-rollout-8d7147: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
  STEP: Dumping all the Cluster API resources in the "md-rollout-g7to6q" namespace @ 01/17/23 18:03:38.7
  STEP: Deleting cluster md-rollout-g7to6q/md-rollout-8d7147 @ 01/17/23 18:03:39.06
  STEP: Deleting cluster md-rollout-8d7147 @ 01/17/23 18:03:39.081
  INFO: Waiting for the Cluster md-rollout-g7to6q/md-rollout-8d7147 to be deleted
  STEP: Waiting for cluster md-rollout-8d7147 to be deleted @ 01/17/23 18:03:39.094
  STEP: Deleting namespace used for hosting the "md-rollout" test spec @ 01/17/23 18:04:09.115
... skipping 57 lines ...
  INFO: Waiting for MachineDeployment rollout for MachineDeploymentTopology "md-0" (class "quick-start-worker") to complete.
  STEP: Deleting a MachineDeploymentTopology in the Cluster Topology and wait for associated MachineDeployment to be deleted @ 01/17/23 18:08:41.108
  INFO: Removing MachineDeploymentTopology from the Cluster Topology.
  INFO: Waiting for MachineDeployment to be deleted.
  STEP: PASSED! @ 01/17/23 18:08:51.193
  STEP: Dumping logs from the "clusterclass-changes-10fzd8" workload cluster @ 01/17/23 18:08:51.193
Failed to get logs for Machine clusterclass-changes-10fzd8-z57vg-qr6gg, Cluster clusterclass-changes-l44ccr/clusterclass-changes-10fzd8: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
  STEP: Dumping all the Cluster API resources in the "clusterclass-changes-l44ccr" namespace @ 01/17/23 18:08:53.387
  STEP: Deleting cluster clusterclass-changes-l44ccr/clusterclass-changes-10fzd8 @ 01/17/23 18:08:53.718
  STEP: Deleting cluster clusterclass-changes-10fzd8 @ 01/17/23 18:08:53.747
  INFO: Waiting for the Cluster clusterclass-changes-l44ccr/clusterclass-changes-10fzd8 to be deleted
  STEP: Waiting for cluster clusterclass-changes-10fzd8 to be deleted @ 01/17/23 18:08:53.76
  STEP: Deleting namespace used for hosting the "clusterclass-changes" test spec @ 01/17/23 18:09:13.776
... skipping 110 lines ...
  Patching MachineHealthCheck unhealthy condition to one of the nodes
  INFO: Patching the node condition to the node
  Waiting for remediation
  Waiting until the node with unhealthy node condition is remediated
  STEP: PASSED! @ 01/17/23 18:19:48.618
  STEP: Dumping logs from the "mhc-remediation-qo9buw" workload cluster @ 01/17/23 18:19:48.618
Failed to get logs for Machine mhc-remediation-qo9buw-md-0-6f7ccc77f9-ppdkq, Cluster mhc-remediation-c4xjx2/mhc-remediation-qo9buw: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
Failed to get logs for Machine mhc-remediation-qo9buw-pwkn6, Cluster mhc-remediation-c4xjx2/mhc-remediation-qo9buw: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
  STEP: Dumping all the Cluster API resources in the "mhc-remediation-c4xjx2" namespace @ 01/17/23 18:19:53.132
  STEP: Deleting cluster mhc-remediation-c4xjx2/mhc-remediation-qo9buw @ 01/17/23 18:19:53.452
  STEP: Deleting cluster mhc-remediation-qo9buw @ 01/17/23 18:19:53.469
  INFO: Waiting for the Cluster mhc-remediation-c4xjx2/mhc-remediation-qo9buw to be deleted
  STEP: Waiting for cluster mhc-remediation-qo9buw to be deleted @ 01/17/23 18:19:53.486
  STEP: Deleting namespace used for hosting the "mhc-remediation" test spec @ 01/17/23 18:20:23.509
... skipping 37 lines ...
  STEP: Waiting for cluster to enter the provisioned phase @ 01/17/23 18:20:24.854
  INFO: Waiting for control plane to be initialized
  INFO: Waiting for the first control plane machine managed by mhc-remediation-av6esl/mhc-remediation-710kz1 to be provisioned
  STEP: Waiting for one control plane node to exist @ 01/17/23 18:21:04.907
  [TIMEDOUT] in [It] - /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/e2e/mhc_remediations.go:116 @ 01/17/23 18:21:23.477
  STEP: Dumping logs from the "mhc-remediation-710kz1" workload cluster @ 01/17/23 18:21:23.479
Failed to get logs for Machine mhc-remediation-710kz1-6rnnb, Cluster mhc-remediation-av6esl/mhc-remediation-710kz1: dialing host IP address at : dial tcp :22: connect: connection refused
Failed to get logs for Machine mhc-remediation-710kz1-md-0-5f67c9665-4nkg5, Cluster mhc-remediation-av6esl/mhc-remediation-710kz1: dialing host IP address at : dial tcp :22: connect: connection refused
  STEP: Dumping all the Cluster API resources in the "mhc-remediation-av6esl" namespace @ 01/17/23 18:21:23.545
  STEP: Deleting cluster mhc-remediation-av6esl/mhc-remediation-710kz1 @ 01/17/23 18:21:23.872
  STEP: Deleting cluster mhc-remediation-710kz1 @ 01/17/23 18:21:23.89
  INFO: Waiting for the Cluster mhc-remediation-av6esl/mhc-remediation-710kz1 to be deleted
  STEP: Waiting for cluster mhc-remediation-710kz1 to be deleted @ 01/17/23 18:21:23.901
  STEP: Deleting namespace used for hosting the "mhc-remediation" test spec @ 01/17/23 18:21:43.914
... skipping 84 lines ...
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs.func2({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc002694030, 0x28}, {0xc002694060, 0x23}, {0xc002a89cf4, 0xb}, ...}, ...}, ...)
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:186
              | out := bufio.NewWriter(f)
              | defer out.Flush()
              > _, err = out.ReadFrom(podLogs)
              | if err != nil && err != io.ErrUnexpectedEOF {
              | 	// Failing to stream logs should not cause the test to fail
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:161
              | 
              | // Watch each container's logs in a goroutine so we can stream them all concurrently.
              > go func(pod corev1.Pod, container corev1.Container) {
              | 	defer GinkgoRecover()
... skipping 29 lines ...
              | for {
              > 	select {
              | 	case <-ctx.Done():
              | 		return
        > sigs.k8s.io/cluster-api/test/framework.WatchPodMetrics
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:225
              | }, retryableOperationTimeout, retryableOperationInterval).Should(Succeed(), "Failed to list Pods for deployment %s", klog.KObj(input.Deployment))
              | 
              > go func() {
              | 	defer GinkgoRecover()
              | 	for {

        goroutine 44092 [sync.Cond.Wait]
... skipping 18 lines ...
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs.func2({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc00173ae80, 0x3a}, {0xc00173aec0, 0x35}, {0xc000cdb9a0, 0x1d}, ...}, ...}, ...)
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:186
              | out := bufio.NewWriter(f)
              | defer out.Flush()
              > _, err = out.ReadFrom(podLogs)
              | if err != nil && err != io.ErrUnexpectedEOF {
              | 	// Failing to stream logs should not cause the test to fail
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:161
              | 
              | // Watch each container's logs in a goroutine so we can stream them all concurrently.
              > go func(pod corev1.Pod, container corev1.Container) {
              | 	defer GinkgoRecover()
... skipping 21 lines ...
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs.func2({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc00173ba40, 0x3e}, {0xc00173ba80, 0x39}, {0xc0023d2420, 0x21}, ...}, ...}, ...)
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:186
              | out := bufio.NewWriter(f)
              | defer out.Flush()
              > _, err = out.ReadFrom(podLogs)
              | if err != nil && err != io.ErrUnexpectedEOF {
              | 	// Failing to stream logs should not cause the test to fail
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:161
              | 
              | // Watch each container's logs in a goroutine so we can stream them all concurrently.
              > go func(pod corev1.Pod, container corev1.Container) {
              | 	defer GinkgoRecover()
... skipping 21 lines ...
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs.func2({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc00215be30, 0x27}, {0xc00215be60, 0x22}, {0xc00235bf44, 0xb}, ...}, ...}, ...)
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:186
              | out := bufio.NewWriter(f)
              | defer out.Flush()
              > _, err = out.ReadFrom(podLogs)
              | if err != nil && err != io.ErrUnexpectedEOF {
              | 	// Failing to stream logs should not cause the test to fail
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:161
              | 
              | // Watch each container's logs in a goroutine so we can stream them all concurrently.
              > go func(pod corev1.Pod, container corev1.Container) {
              | 	defer GinkgoRecover()
... skipping 6 lines ...
              | for {
              > 	select {
              | 	case <-ctx.Done():
              | 		return
        > sigs.k8s.io/cluster-api/test/framework.WatchPodMetrics
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:225
              | }, retryableOperationTimeout, retryableOperationInterval).Should(Succeed(), "Failed to list Pods for deployment %s", klog.KObj(input.Deployment))
              | 
              > go func() {
              | 	defer GinkgoRecover()
              | 	for {

        goroutine 44115 [select]
... skipping 3 lines ...
              | for {
              > 	select {
              | 	case <-ctx.Done():
              | 		return
        > sigs.k8s.io/cluster-api/test/framework.WatchPodMetrics
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:225
              | }, retryableOperationTimeout, retryableOperationInterval).Should(Succeed(), "Failed to list Pods for deployment %s", klog.KObj(input.Deployment))
              | 
              > go func() {
              | 	defer GinkgoRecover()
              | 	for {

        goroutine 44146 [select]
... skipping 3 lines ...
              | for {
              > 	select {
              | 	case <-ctx.Done():
              | 		return
        > sigs.k8s.io/cluster-api/test/framework.WatchPodMetrics
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:225
              | }, retryableOperationTimeout, retryableOperationInterval).Should(Succeed(), "Failed to list Pods for deployment %s", klog.KObj(input.Deployment))
              | 
              > go func() {
              | 	defer GinkgoRecover()
              | 	for {
------------------------------
S [SKIPPED]
... skipping 17 lines ...
/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/e2e/node_drain_timeout.go:83
------------------------------
[SynchronizedAfterSuite] 
/home/prow/go/src/sigs.k8s.io/cluster-api-provider-vsphere/test/e2e/e2e_suite_test.go:159
  STEP: Cleaning up the vSphere session @ 01/17/23 18:21:43.938
  STEP: Tearing down the management cluster @ 01/17/23 18:21:44.115
  INFO: Failed to list the machines: Get "https://127.0.0.1:36063/apis/cluster.x-k8s.io/v1beta1/namespaces/mhc-remediation-av6esl/machines?labelSelector=cluster.x-k8s.io%2Fcluster-name%3Dmhc-remediation-710kz1%2Ccluster.x-k8s.io%2Fcontrol-plane%3D": dial tcp 127.0.0.1:36063: connect: connection refused
[SynchronizedAfterSuite] PASSED [1.825 seconds]
------------------------------

Summarizing 1 Failure:
  [TIMEDOUT] When testing unhealthy machines remediation [It] Should successfully trigger KCP remediation
  /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/e2e/mhc_remediations.go:116

Ran 11 of 17 Specs in 3559.719 seconds
FAIL! - Suite Timeout Elapsed -- 10 Passed | 1 Failed | 1 Pending | 5 Skipped
--- FAIL: TestE2E (3559.72s)
FAIL

Ginkgo ran 1 suite in 1h0m22.390316483s

Test Suite Failed

real	60m22.414s
user	6m31.758s
sys	1m23.240s
make: *** [Makefile:183: e2e] Error 1
Releasing IP claims
ipclaim.ipam.metal3.io "ip-claim-ba37d82e0f1fd930207a609c5241d04eb76eb735" deleted
ipclaim.ipam.metal3.io "workload-ip-claim-b71ef088328c0253ab399bbca88745206e4e1093" deleted
vpn
WARNING: [capv-prow@cluster-api-provider-vsphere.iam.gserviceaccount.com] appears to be a service account. Service account tokens cannot be revoked, but they will expire automatically. To prevent use of the service account token earlier than the expiration, delete or disable the parent service account. To explicitly delete the key associated with the service account use `gcloud iam service-accounts keys delete` instead`.
Revoked credentials:
... skipping 13 lines ...