periodic-cluster-api-provider-vsphere-e2e-main #1616303606024114176
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 0 failed / 0 succeeded |
| Started | |
| Elapsed | 1h5m |
| Revision | main |
... skipping 568 lines ... INFO: Waiting for the machine deployments to be provisioned [1mSTEP:[0m Waiting for the workload nodes to exist [38;5;243m@ 01/20/23 05:26:00.012[0m [1mSTEP:[0m Checking all the machines controlled by quick-start-63audh-md-0-zmhk5 are in the "<None>" failure domain [38;5;243m@ 01/20/23 05:27:30.144[0m INFO: Waiting for the machine pools to be provisioned [1mSTEP:[0m PASSED! [38;5;243m@ 01/20/23 05:27:30.186[0m [1mSTEP:[0m Dumping logs from the "quick-start-63audh" workload cluster [38;5;243m@ 01/20/23 05:27:30.186[0m Failed to get logs for Machine quick-start-63audh-md-0-zmhk5-698c7bb6dd-j5ldn, Cluster quick-start-wks7ot/quick-start-63audh: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 Failed to get logs for Machine quick-start-63audh-pfmx5-x5ht9, Cluster quick-start-wks7ot/quick-start-63audh: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 [1mSTEP:[0m Dumping all the Cluster API resources in the "quick-start-wks7ot" namespace [38;5;243m@ 01/20/23 05:27:34.669[0m [1mSTEP:[0m Deleting cluster quick-start-wks7ot/quick-start-63audh [38;5;243m@ 01/20/23 05:27:34.951[0m [1mSTEP:[0m Deleting cluster quick-start-63audh [38;5;243m@ 01/20/23 05:27:34.971[0m INFO: Waiting for the Cluster quick-start-wks7ot/quick-start-63audh to be deleted [1mSTEP:[0m Waiting for cluster quick-start-63audh to be deleted [38;5;243m@ 01/20/23 05:27:34.985[0m [1mSTEP:[0m Deleting namespace used for hosting the "quick-start" test spec [38;5;243m@ 01/20/23 05:28:05.007[0m ... skipping 52 lines ... Patching MachineHealthCheck unhealthy condition to one of the nodes INFO: Patching the node condition to the node Waiting for remediation Waiting until the node with unhealthy node condition is remediated [1mSTEP:[0m PASSED! [38;5;243m@ 01/20/23 05:33:41.394[0m [1mSTEP:[0m Dumping logs from the "mhc-remediation-z7sddb" workload cluster [38;5;243m@ 01/20/23 05:33:41.394[0m Failed to get logs for Machine mhc-remediation-z7sddb-md-0-7c767ff646-bfbgp, Cluster mhc-remediation-g9e2lr/mhc-remediation-z7sddb: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 Failed to get logs for Machine mhc-remediation-z7sddb-tsj6t, Cluster mhc-remediation-g9e2lr/mhc-remediation-z7sddb: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 [1mSTEP:[0m Dumping all the Cluster API resources in the "mhc-remediation-g9e2lr" namespace [38;5;243m@ 01/20/23 05:33:45.723[0m [1mSTEP:[0m Deleting cluster mhc-remediation-g9e2lr/mhc-remediation-z7sddb [38;5;243m@ 01/20/23 05:33:46.025[0m [1mSTEP:[0m Deleting cluster mhc-remediation-z7sddb [38;5;243m@ 01/20/23 05:33:46.046[0m INFO: Waiting for the Cluster mhc-remediation-g9e2lr/mhc-remediation-z7sddb to be deleted [1mSTEP:[0m Waiting for cluster mhc-remediation-z7sddb to be deleted [38;5;243m@ 01/20/23 05:33:46.06[0m [1mSTEP:[0m Deleting namespace used for hosting the "mhc-remediation" test spec [38;5;243m@ 01/20/23 05:34:16.085[0m ... skipping 54 lines ... Patching MachineHealthCheck unhealthy condition to one of the nodes INFO: Patching the node condition to the node Waiting for remediation Waiting until the node with unhealthy node condition is remediated [1mSTEP:[0m PASSED! [38;5;243m@ 01/20/23 05:43:36.741[0m [1mSTEP:[0m Dumping logs from the "mhc-remediation-rh8hi5" workload cluster [38;5;243m@ 01/20/23 05:43:36.742[0m Failed to get logs for Machine mhc-remediation-rh8hi5-md-0-7d57795685-s7mth, Cluster mhc-remediation-tpqjdp/mhc-remediation-rh8hi5: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 Failed to get logs for Machine mhc-remediation-rh8hi5-vbpg5, Cluster mhc-remediation-tpqjdp/mhc-remediation-rh8hi5: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 Failed to get logs for Machine mhc-remediation-rh8hi5-vndtf, Cluster mhc-remediation-tpqjdp/mhc-remediation-rh8hi5: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 Failed to get logs for Machine mhc-remediation-rh8hi5-zkl55, Cluster mhc-remediation-tpqjdp/mhc-remediation-rh8hi5: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 [1mSTEP:[0m Dumping all the Cluster API resources in the "mhc-remediation-tpqjdp" namespace [38;5;243m@ 01/20/23 05:43:44.693[0m [1mSTEP:[0m Deleting cluster mhc-remediation-tpqjdp/mhc-remediation-rh8hi5 [38;5;243m@ 01/20/23 05:43:45[0m [1mSTEP:[0m Deleting cluster mhc-remediation-rh8hi5 [38;5;243m@ 01/20/23 05:43:45.023[0m INFO: Waiting for the Cluster mhc-remediation-tpqjdp/mhc-remediation-rh8hi5 to be deleted [1mSTEP:[0m Waiting for cluster mhc-remediation-rh8hi5 to be deleted [38;5;243m@ 01/20/23 05:43:45.036[0m [1mSTEP:[0m Deleting namespace used for hosting the "mhc-remediation" test spec [38;5;243m@ 01/20/23 05:44:35.065[0m ... skipping 44 lines ... INFO: Waiting for the machine deployments to be provisioned [1mSTEP:[0m Waiting for the workload nodes to exist [38;5;243m@ 01/20/23 05:47:36.344[0m [1mSTEP:[0m Checking all the machines controlled by quick-start-5p7vyg-md-0 are in the "<None>" failure domain [38;5;243m@ 01/20/23 05:48:36.425[0m INFO: Waiting for the machine pools to be provisioned [1mSTEP:[0m PASSED! [38;5;243m@ 01/20/23 05:48:36.467[0m [1mSTEP:[0m Dumping logs from the "quick-start-5p7vyg" workload cluster [38;5;243m@ 01/20/23 05:48:36.468[0m Failed to get logs for Machine quick-start-5p7vyg-2d9bb, Cluster quick-start-zy396e/quick-start-5p7vyg: dialing host IP address at 192.168.6.26: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain Failed to get logs for Machine quick-start-5p7vyg-md-0-59c459c7bf-ts774, Cluster quick-start-zy396e/quick-start-5p7vyg: dialing host IP address at 192.168.6.15: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain [1mSTEP:[0m Dumping all the Cluster API resources in the "quick-start-zy396e" namespace [38;5;243m@ 01/20/23 05:48:38.807[0m [1mSTEP:[0m Deleting cluster quick-start-zy396e/quick-start-5p7vyg [38;5;243m@ 01/20/23 05:48:39.09[0m [1mSTEP:[0m Deleting cluster quick-start-5p7vyg [38;5;243m@ 01/20/23 05:48:39.109[0m INFO: Waiting for the Cluster quick-start-zy396e/quick-start-5p7vyg to be deleted [1mSTEP:[0m Waiting for cluster quick-start-5p7vyg to be deleted [38;5;243m@ 01/20/23 05:48:39.121[0m [1mSTEP:[0m Deleting namespace used for hosting the "quick-start" test spec [38;5;243m@ 01/20/23 05:49:09.14[0m ... skipping 50 lines ... INFO: Waiting for rolling upgrade to start. INFO: Waiting for MachineDeployment rolling upgrade to start INFO: Waiting for rolling upgrade to complete. INFO: Waiting for MachineDeployment rolling upgrade to complete [1mSTEP:[0m PASSED! [38;5;243m@ 01/20/23 05:56:10.591[0m [1mSTEP:[0m Dumping logs from the "md-rollout-obxgah" workload cluster [38;5;243m@ 01/20/23 05:56:10.591[0m Failed to get logs for Machine md-rollout-obxgah-9lcnp, Cluster md-rollout-3j19db/md-rollout-obxgah: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 Failed to get logs for Machine md-rollout-obxgah-md-0-6546998cf8-xdgzg, Cluster md-rollout-3j19db/md-rollout-obxgah: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 [1mSTEP:[0m Dumping all the Cluster API resources in the "md-rollout-3j19db" namespace [38;5;243m@ 01/20/23 05:56:15.149[0m [1mSTEP:[0m Deleting cluster md-rollout-3j19db/md-rollout-obxgah [38;5;243m@ 01/20/23 05:56:15.44[0m [1mSTEP:[0m Deleting cluster md-rollout-obxgah [38;5;243m@ 01/20/23 05:56:15.46[0m INFO: Waiting for the Cluster md-rollout-3j19db/md-rollout-obxgah to be deleted [1mSTEP:[0m Waiting for cluster md-rollout-obxgah to be deleted [38;5;243m@ 01/20/23 05:56:15.472[0m [1mSTEP:[0m Deleting namespace used for hosting the "md-rollout" test spec [38;5;243m@ 01/20/23 05:56:45.49[0m ... skipping 50 lines ... INFO: Waiting for correct number of replicas to exist [1mSTEP:[0m Scaling the MachineDeployment down to 1 [38;5;243m@ 01/20/23 06:02:27.356[0m INFO: Scaling machine deployment md-scale-8qpfyw/md-scale-ce2vqt-md-0 from 3 to 1 replicas INFO: Waiting for correct number of replicas to exist [1mSTEP:[0m PASSED! [38;5;243m@ 01/20/23 06:02:37.475[0m [1mSTEP:[0m Dumping logs from the "md-scale-ce2vqt" workload cluster [38;5;243m@ 01/20/23 06:02:37.475[0m Failed to get logs for Machine md-scale-ce2vqt-d5wqr, Cluster md-scale-8qpfyw/md-scale-ce2vqt: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 Failed to get logs for Machine md-scale-ce2vqt-md-0-659f97c996-zjgk6, Cluster md-scale-8qpfyw/md-scale-ce2vqt: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 [1mSTEP:[0m Dumping all the Cluster API resources in the "md-scale-8qpfyw" namespace [38;5;243m@ 01/20/23 06:02:41.972[0m [1mSTEP:[0m Deleting cluster md-scale-8qpfyw/md-scale-ce2vqt [38;5;243m@ 01/20/23 06:02:42.298[0m [1mSTEP:[0m Deleting cluster md-scale-ce2vqt [38;5;243m@ 01/20/23 06:02:42.316[0m INFO: Waiting for the Cluster md-scale-8qpfyw/md-scale-ce2vqt to be deleted [1mSTEP:[0m Waiting for cluster md-scale-ce2vqt to be deleted [38;5;243m@ 01/20/23 06:02:42.329[0m [1mSTEP:[0m Deleting namespace used for hosting the "md-scale" test spec [38;5;243m@ 01/20/23 06:03:12.351[0m ... skipping 168 lines ... [1mSTEP:[0m Waiting for deployment node-drain-rd57fs-unevictable-workload/unevictable-pod-clx to be available [38;5;243m@ 01/20/23 06:19:35.024[0m [1mSTEP:[0m Scale down the controlplane of the workload cluster and make sure that nodes running workload can be deleted even the draining process is blocked. [38;5;243m@ 01/20/23 06:19:45.338[0m INFO: Scaling controlplane node-drain-rd57fs/node-drain-kqcybf from 3 to 1 replicas INFO: Waiting for correct number of replicas to exist [38;5;214m[TIMEDOUT][0m in [It] - /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/e2e/node_drain_timeout.go:83 [38;5;243m@ 01/20/23 06:20:33.46[0m [1mSTEP:[0m Dumping logs from the "node-drain-kqcybf" workload cluster [38;5;243m@ 01/20/23 06:20:33.461[0m Failed to get logs for Machine node-drain-kqcybf-6wsfg, Cluster node-drain-rd57fs/node-drain-kqcybf: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 Failed to get logs for Machine node-drain-kqcybf-qpsfk, Cluster node-drain-rd57fs/node-drain-kqcybf: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 Failed to get logs for Machine node-drain-kqcybf-rh495, Cluster node-drain-rd57fs/node-drain-kqcybf: running command "cat /var/log/cloud-init-output.log": Process exited with status 1 [1mSTEP:[0m Dumping all the Cluster API resources in the "node-drain-rd57fs" namespace [38;5;243m@ 01/20/23 06:20:39.065[0m [1mSTEP:[0m Deleting cluster node-drain-rd57fs/node-drain-kqcybf [38;5;243m@ 01/20/23 06:20:39.359[0m [1mSTEP:[0m Deleting cluster node-drain-kqcybf [38;5;243m@ 01/20/23 06:20:39.381[0m INFO: Waiting for the Cluster node-drain-rd57fs/node-drain-kqcybf to be deleted [1mSTEP:[0m Waiting for cluster node-drain-kqcybf to be deleted [38;5;243m@ 01/20/23 06:20:39.395[0m [38;5;214m[TIMEDOUT][0m in [AfterEach] - /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/e2e/node_drain_timeout.go:155 [38;5;243m@ 01/20/23 06:21:03.462[0m ... skipping 52 lines ... [38;5;9m[1mSummarizing 1 Failure:[0m [38;5;214m[TIMEDOUT][0m [0mWhen testing node drain timeout [38;5;214m[1m[It] A node should be forcefully removed if it cannot be drained in time[0m [38;5;243m/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.3.0/e2e/node_drain_timeout.go:83[0m [38;5;9m[1mRan 9 of 17 Specs in 3579.236 seconds[0m [38;5;9m[1mFAIL! - Suite Timeout Elapsed[0m -- [38;5;10m[1m8 Passed[0m | [38;5;9m[1m1 Failed[0m | [38;5;11m[1m1 Pending[0m | [38;5;14m[1m7 Skipped[0m --- FAIL: TestE2E (3579.24s) FAIL Ginkgo ran 1 suite in 1h0m31.857139553s Test Suite Failed real 60m31.879s user 5m34.973s sys 1m6.665s make: *** [Makefile:183: e2e] Error 1 Releasing IP claims ipclaim.ipam.metal3.io "ip-claim-410d37f524af31fef9b0730e233ffc87ff5d8a7c" deleted ipclaim.ipam.metal3.io "workload-ip-claim-3013fc460deb436aeb1e5268382f7d8c9d73a8b4" deleted vpn WARNING: [capv-prow@cluster-api-provider-vsphere.iam.gserviceaccount.com] appears to be a service account. Service account tokens cannot be revoked, but they will expire automatically. To prevent use of the service account token earlier than the expiration, delete or disable the parent service account. To explicitly delete the key associated with the service account use `gcloud iam service-accounts keys delete` instead`. Revoked credentials: ... skipping 13 lines ...