This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 0 failed / 0 succeeded
Started2023-01-26 17:21
Elapsed1h6m
Revisionmain

No Test Failures!

Error lines from build-log.txt

... skipping 602 lines ...
  INFO: Waiting for correct number of replicas to exist
  STEP: Scaling the MachineDeployment down to 1 @ 01/26/23 17:34:40.89
  INFO: Scaling machine deployment md-scale-pm75md/md-scale-a5nfa5-md-0 from 3 to 1 replicas
  INFO: Waiting for correct number of replicas to exist
  STEP: PASSED! @ 01/26/23 17:34:51.07
  STEP: Dumping logs from the "md-scale-a5nfa5" workload cluster @ 01/26/23 17:34:51.07
Failed to get logs for Machine md-scale-a5nfa5-5sxtx, Cluster md-scale-pm75md/md-scale-a5nfa5: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
Failed to get logs for Machine md-scale-a5nfa5-md-0-59bfddc5c-vmp4g, Cluster md-scale-pm75md/md-scale-a5nfa5: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
  STEP: Dumping all the Cluster API resources in the "md-scale-pm75md" namespace @ 01/26/23 17:34:55.605
  STEP: Deleting cluster md-scale-pm75md/md-scale-a5nfa5 @ 01/26/23 17:34:55.933
  STEP: Deleting cluster md-scale-a5nfa5 @ 01/26/23 17:34:55.955
  INFO: Waiting for the Cluster md-scale-pm75md/md-scale-a5nfa5 to be deleted
  STEP: Waiting for cluster md-scale-a5nfa5 to be deleted @ 01/26/23 17:34:55.969
  STEP: Deleting namespace used for hosting the "md-scale" test spec @ 01/26/23 17:35:25.991
... skipping 58 lines ...
  STEP: Waiting for deployment node-drain-80g6es-unevictable-workload/unevictable-pod-pyy to be available @ 01/26/23 17:43:13.198
  STEP: Scale down the controlplane of the workload cluster and make sure that nodes running workload can be deleted even the draining process is blocked. @ 01/26/23 17:43:23.513
  INFO: Scaling controlplane node-drain-80g6es/node-drain-3gkipn from 3 to 1 replicas
  INFO: Waiting for correct number of replicas to exist
  STEP: PASSED! @ 01/26/23 17:47:04.131
  STEP: Dumping logs from the "node-drain-3gkipn" workload cluster @ 01/26/23 17:47:04.131
Failed to get logs for Machine node-drain-3gkipn-4mqt2, Cluster node-drain-80g6es/node-drain-3gkipn: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
  STEP: Dumping all the Cluster API resources in the "node-drain-80g6es" namespace @ 01/26/23 17:47:06.175
  STEP: Deleting cluster node-drain-80g6es/node-drain-3gkipn @ 01/26/23 17:47:06.496
  STEP: Deleting cluster node-drain-3gkipn @ 01/26/23 17:47:06.516
  INFO: Waiting for the Cluster node-drain-80g6es/node-drain-3gkipn to be deleted
  STEP: Waiting for cluster node-drain-3gkipn to be deleted @ 01/26/23 17:47:06.53
  STEP: Deleting namespace used for hosting the "node-drain" test spec @ 01/26/23 17:47:36.552
... skipping 57 lines ...
  INFO: Waiting for MachineDeployment rollout for MachineDeploymentTopology "md-0" (class "quick-start-worker") to complete.
  STEP: Deleting a MachineDeploymentTopology in the Cluster Topology and wait for associated MachineDeployment to be deleted @ 01/26/23 17:51:58.736
  INFO: Removing MachineDeploymentTopology from the Cluster Topology.
  INFO: Waiting for MachineDeployment to be deleted.
  STEP: PASSED! @ 01/26/23 17:52:08.82
  STEP: Dumping logs from the "clusterclass-changes-cevt8c" workload cluster @ 01/26/23 17:52:08.82
Failed to get logs for Machine clusterclass-changes-cevt8c-f6zmh-hnpwk, Cluster clusterclass-changes-zzx3v1/clusterclass-changes-cevt8c: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
  STEP: Dumping all the Cluster API resources in the "clusterclass-changes-zzx3v1" namespace @ 01/26/23 17:52:10.71
  STEP: Deleting cluster clusterclass-changes-zzx3v1/clusterclass-changes-cevt8c @ 01/26/23 17:52:11.008
  STEP: Deleting cluster clusterclass-changes-cevt8c @ 01/26/23 17:52:11.03
  INFO: Waiting for the Cluster clusterclass-changes-zzx3v1/clusterclass-changes-cevt8c to be deleted
  STEP: Waiting for cluster clusterclass-changes-cevt8c to be deleted @ 01/26/23 17:52:11.042
  STEP: Deleting namespace used for hosting the "clusterclass-changes" test spec @ 01/26/23 17:52:31.061
... skipping 166 lines ...
  INFO: Waiting for the machine deployments to be provisioned
  STEP: Waiting for the workload nodes to exist @ 01/26/23 18:07:54.682
  STEP: Checking all the machines controlled by quick-start-12lhq7-md-0 are in the "<None>" failure domain @ 01/26/23 18:08:54.768
  INFO: Waiting for the machine pools to be provisioned
  STEP: PASSED! @ 01/26/23 18:08:54.817
  STEP: Dumping logs from the "quick-start-12lhq7" workload cluster @ 01/26/23 18:08:54.818
Failed to get logs for Machine quick-start-12lhq7-cfwfm, Cluster quick-start-ket1q5/quick-start-12lhq7: dialing host IP address at 192.168.6.83: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Failed to get logs for Machine quick-start-12lhq7-md-0-7774cc8-5bsl6, Cluster quick-start-ket1q5/quick-start-12lhq7: dialing host IP address at 192.168.6.110: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
  STEP: Dumping all the Cluster API resources in the "quick-start-ket1q5" namespace @ 01/26/23 18:08:57.201
  STEP: Deleting cluster quick-start-ket1q5/quick-start-12lhq7 @ 01/26/23 18:08:57.56
  STEP: Deleting cluster quick-start-12lhq7 @ 01/26/23 18:08:57.577
  INFO: Waiting for the Cluster quick-start-ket1q5/quick-start-12lhq7 to be deleted
  STEP: Waiting for cluster quick-start-12lhq7 to be deleted @ 01/26/23 18:08:57.592
  STEP: Deleting namespace used for hosting the "quick-start" test spec @ 01/26/23 18:09:27.613
... skipping 44 lines ...
  INFO: Waiting for the machine deployments to be provisioned
  STEP: Waiting for the workload nodes to exist @ 01/26/23 18:11:59.111
  STEP: Checking all the machines controlled by quick-start-6wdc9h-md-0-fs2dh are in the "<None>" failure domain @ 01/26/23 18:12:59.195
  INFO: Waiting for the machine pools to be provisioned
  STEP: PASSED! @ 01/26/23 18:12:59.246
  STEP: Dumping logs from the "quick-start-6wdc9h" workload cluster @ 01/26/23 18:12:59.246
Failed to get logs for Machine quick-start-6wdc9h-jw94z-59zvm, Cluster quick-start-vo2ifz/quick-start-6wdc9h: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
Failed to get logs for Machine quick-start-6wdc9h-md-0-fs2dh-7666d66fd9-fwvf8, Cluster quick-start-vo2ifz/quick-start-6wdc9h: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
  STEP: Dumping all the Cluster API resources in the "quick-start-vo2ifz" namespace @ 01/26/23 18:13:04.182
  STEP: Deleting cluster quick-start-vo2ifz/quick-start-6wdc9h @ 01/26/23 18:13:04.494
  STEP: Deleting cluster quick-start-6wdc9h @ 01/26/23 18:13:04.516
  INFO: Waiting for the Cluster quick-start-vo2ifz/quick-start-6wdc9h to be deleted
  STEP: Waiting for cluster quick-start-6wdc9h to be deleted @ 01/26/23 18:13:04.533
  STEP: Deleting namespace used for hosting the "quick-start" test spec @ 01/26/23 18:13:34.557
... skipping 56 lines ...
  Patching MachineHealthCheck unhealthy condition to one of the nodes
  INFO: Patching the node condition to the node
  Waiting for remediation
  Waiting until the node with unhealthy node condition is remediated
  STEP: PASSED! @ 01/26/23 18:19:31.146
  STEP: Dumping logs from the "mhc-remediation-2hzpyw" workload cluster @ 01/26/23 18:19:31.146
Failed to get logs for Machine mhc-remediation-2hzpyw-bj9rb, Cluster mhc-remediation-uedokg/mhc-remediation-2hzpyw: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
Failed to get logs for Machine mhc-remediation-2hzpyw-md-0-764dc89b6d-tw5cc, Cluster mhc-remediation-uedokg/mhc-remediation-2hzpyw: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
  STEP: Dumping all the Cluster API resources in the "mhc-remediation-uedokg" namespace @ 01/26/23 18:19:35.285
  STEP: Deleting cluster mhc-remediation-uedokg/mhc-remediation-2hzpyw @ 01/26/23 18:19:35.595
  STEP: Deleting cluster mhc-remediation-2hzpyw @ 01/26/23 18:19:35.616
  INFO: Waiting for the Cluster mhc-remediation-uedokg/mhc-remediation-2hzpyw to be deleted
  STEP: Waiting for cluster mhc-remediation-2hzpyw to be deleted @ 01/26/23 18:19:35.64
  STEP: Deleting namespace used for hosting the "mhc-remediation" test spec @ 01/26/23 18:20:05.665
... skipping 40 lines ...
  STEP: Waiting for one control plane node to exist @ 01/26/23 18:20:56.984
  INFO: Waiting for control plane to be ready
  INFO: Waiting for the remaining control plane machines managed by mhc-remediation-b1qt8h/mhc-remediation-giwm8g to be provisioned
  STEP: Waiting for all control plane nodes to exist @ 01/26/23 18:22:57.083
  [TIMEDOUT] in [It] - /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/e2e/mhc_remediations.go:116 @ 01/26/23 18:26:09.24
  STEP: Dumping logs from the "mhc-remediation-giwm8g" workload cluster @ 01/26/23 18:26:09.242
Failed to get logs for Machine mhc-remediation-giwm8g-6q5hb, Cluster mhc-remediation-b1qt8h/mhc-remediation-giwm8g: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
Failed to get logs for Machine mhc-remediation-giwm8g-l9bp8, Cluster mhc-remediation-b1qt8h/mhc-remediation-giwm8g: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
Failed to get logs for Machine mhc-remediation-giwm8g-md-0-5d4ccd7b5b-pd5k6, Cluster mhc-remediation-b1qt8h/mhc-remediation-giwm8g: running command "cat /var/log/cloud-init-output.log": Process exited with status 1
Failed to get logs for Machine mhc-remediation-giwm8g-mzg6s, Cluster mhc-remediation-b1qt8h/mhc-remediation-giwm8g: dialing host IP address at : dial tcp :22: connect: connection refused
  STEP: Dumping all the Cluster API resources in the "mhc-remediation-b1qt8h" namespace @ 01/26/23 18:26:15.581
  STEP: Deleting cluster mhc-remediation-b1qt8h/mhc-remediation-giwm8g @ 01/26/23 18:26:16.016
  STEP: Deleting cluster mhc-remediation-giwm8g @ 01/26/23 18:26:16.037
  INFO: Waiting for the Cluster mhc-remediation-b1qt8h/mhc-remediation-giwm8g to be deleted
  STEP: Waiting for cluster mhc-remediation-giwm8g to be deleted @ 01/26/23 18:26:16.054
  INFO: Waiting for control plane mhc-remediation-b1qt8h/mhc-remediation-giwm8g to be ready (implies underlying nodes to be ready as well)
... skipping 78 lines ...
              | for {
              > 	select {
              | 	case <-ctx.Done():
              | 		return
        > sigs.k8s.io/cluster-api/test/framework.WatchPodMetrics
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:225
              | }, retryableOperationTimeout, retryableOperationInterval).Should(Succeed(), "Failed to list Pods for deployment %s", klog.KObj(input.Deployment))
              | 
              > go func() {
              | 	defer GinkgoRecover()
              | 	for {

        goroutine 23259 [select]
... skipping 3 lines ...
              | for {
              > 	select {
              | 	case <-ctx.Done():
              | 		return
        > sigs.k8s.io/cluster-api/test/framework.WatchPodMetrics
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:225
              | }, retryableOperationTimeout, retryableOperationInterval).Should(Succeed(), "Failed to list Pods for deployment %s", klog.KObj(input.Deployment))
              | 
              > go func() {
              | 	defer GinkgoRecover()
              | 	for {

        goroutine 23245 [sync.Cond.Wait]
... skipping 18 lines ...
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs.func2({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc00288f2c0, 0x28}, {0xc00288f2f0, 0x23}, {0xc0027fb4c0, 0xb}, ...}, ...}, ...)
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:186
              | out := bufio.NewWriter(f)
              | defer out.Flush()
              > _, err = out.ReadFrom(podLogs)
              | if err != nil && err != io.ErrUnexpectedEOF {
              | 	// Failing to stream logs should not cause the test to fail
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:161
              | 
              | // Watch each container's logs in a goroutine so we can stream them all concurrently.
              > go func(pod corev1.Pod, container corev1.Container) {
              | 	defer GinkgoRecover()
... skipping 21 lines ...
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs.func2({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc0021d7680, 0x27}, {0xc0021d76b0, 0x22}, {0xc0022b3b20, 0xb}, ...}, ...}, ...)
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:186
              | out := bufio.NewWriter(f)
              | defer out.Flush()
              > _, err = out.ReadFrom(podLogs)
              | if err != nil && err != io.ErrUnexpectedEOF {
              | 	// Failing to stream logs should not cause the test to fail
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:161
              | 
              | // Watch each container's logs in a goroutine so we can stream them all concurrently.
              > go func(pod corev1.Pod, container corev1.Container) {
              | 	defer GinkgoRecover()
... skipping 29 lines ...
              | for {
              > 	select {
              | 	case <-ctx.Done():
              | 		return
        > sigs.k8s.io/cluster-api/test/framework.WatchPodMetrics
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:225
              | }, retryableOperationTimeout, retryableOperationInterval).Should(Succeed(), "Failed to list Pods for deployment %s", klog.KObj(input.Deployment))
              | 
              > go func() {
              | 	defer GinkgoRecover()
              | 	for {

        goroutine 23251 [sync.Cond.Wait]
... skipping 18 lines ...
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs.func2({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc0015c6c00, 0x3e}, {0xc0015c6c40, 0x39}, {0xc00220c690, 0x21}, ...}, ...}, ...)
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:186
              | out := bufio.NewWriter(f)
              | defer out.Flush()
              > _, err = out.ReadFrom(podLogs)
              | if err != nil && err != io.ErrUnexpectedEOF {
              | 	// Failing to stream logs should not cause the test to fail
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:161
              | 
              | // Watch each container's logs in a goroutine so we can stream them all concurrently.
              > go func(pod corev1.Pod, container corev1.Container) {
              | 	defer GinkgoRecover()
... skipping 21 lines ...
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs.func2({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc000738640, 0x3a}, {0xc000738680, 0x35}, {0xc000a82aa0, 0x1d}, ...}, ...}, ...)
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:186
              | out := bufio.NewWriter(f)
              | defer out.Flush()
              > _, err = out.ReadFrom(podLogs)
              | if err != nil && err != io.ErrUnexpectedEOF {
              | 	// Failing to stream logs should not cause the test to fail
        > sigs.k8s.io/cluster-api/test/framework.WatchDeploymentLogs
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:161
              | 
              | // Watch each container's logs in a goroutine so we can stream them all concurrently.
              > go func(pod corev1.Pod, container corev1.Container) {
              | 	defer GinkgoRecover()
... skipping 6 lines ...
              | for {
              > 	select {
              | 	case <-ctx.Done():
              | 		return
        > sigs.k8s.io/cluster-api/test/framework.WatchPodMetrics
            /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/framework/deployment_helpers.go:225
              | }, retryableOperationTimeout, retryableOperationInterval).Should(Succeed(), "Failed to list Pods for deployment %s", klog.KObj(input.Deployment))
              | 
              > go func() {
              | 	defer GinkgoRecover()
              | 	for {

  There were additional failures detected.  To view them in detail run ginkgo -vv
... skipping 9 lines ...

Summarizing 1 Failure:
  [TIMEDOUT] When testing unhealthy machines remediation [It] Should successfully trigger KCP remediation
  /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/e2e/mhc_remediations.go:116

Ran 9 of 17 Specs in 3574.738 seconds
FAIL! - Suite Timeout Elapsed -- 8 Passed | 1 Failed | 1 Pending | 7 Skipped
--- FAIL: TestE2E (3574.74s)
FAIL

Ginkgo ran 1 suite in 1h0m31.953156524s

Test Suite Failed

real	60m31.975s
user	5m56.966s
sys	1m13.980s
make: *** [Makefile:183: e2e] Error 1
Releasing IP claims
ipclaim.ipam.metal3.io "ip-claim-2d0624539feadc2818afb33bd65e3f05daacf0ac" deleted
ipclaim.ipam.metal3.io "workload-ip-claim-9ccde7bcda4738186ad80d764310a72b481df39b" deleted
vpn
WARNING: [capv-prow@cluster-api-provider-vsphere.iam.gserviceaccount.com] appears to be a service account. Service account tokens cannot be revoked, but they will expire automatically. To prevent use of the service account token earlier than the expiration, delete or disable the parent service account. To explicitly delete the key associated with the service account use `gcloud iam service-accounts keys delete` instead`.
Revoked credentials:
... skipping 13 lines ...