periodic-cluster-api-provider-azure-capi-e2e-v1beta1-minus-1 #1615031026914955264
Recent runs || View in Spyglass
Recent runs || View in Spyglass
| Result | FAILURE |
| Tests | 0 failed / 6 succeeded |
| Started | |
| Elapsed | 4h15m |
| Revision | release-1.6 |
capz-e2e Running the Cluster API E2E tests Running the self-hosted spec Should pivot the bootstrap cluster to a self-hosted cluster
capz-e2e Running the Cluster API E2E tests Should successfully exercise machine pools Should successfully create a cluster with machine pool machines
capz-e2e Running the Cluster API E2E tests Should successfully remediate unhealthy machines with MachineHealthCheck Should successfully trigger KCP remediation
capz-e2e Running the Cluster API E2E tests Should successfully remediate unhealthy machines with MachineHealthCheck Should successfully trigger machine deployment remediation
capz-e2e Running the Cluster API E2E tests Should successfully scale out and scale in a MachineDeployment Should successfully scale a MachineDeployment up and down upon changes to the MachineDeployment replica count
capz-e2e Running the Cluster API E2E tests Should successfully set and use node drain timeout A node should be forcefully removed if it cannot be drained in time
capz-e2e Conformance Tests conformance-tests
capz-e2e Running the Cluster API E2E tests API Version Upgrade upgrade from v1alpha4 to v1beta1, and scale workload clusters created in v1alpha4 Should create a management cluster and then upgrade all the providers
capz-e2e Running the Cluster API E2E tests Running KCP upgrade in a HA cluster [K8s-Upgrade] Should create and upgrade a workload cluster and eventually run kubetest
capz-e2e Running the Cluster API E2E tests Running KCP upgrade in a HA cluster using scale in rollout [K8s-Upgrade] Should create and upgrade a workload cluster and eventually run kubetest
capz-e2e Running the Cluster API E2E tests Running the workload cluster upgrade spec [K8s-Upgrade] Should create and upgrade a workload cluster and eventually run kubetest
capz-e2e Workload cluster creation Creating a GPU-enabled cluster [OPTIONAL] with a single control plane node and 1 node
capz-e2e Workload cluster creation Creating a VMSS cluster [REQUIRED] with a single control plane node and an AzureMachinePool with 2 Linux and 2 Windows worker nodes
capz-e2e Workload cluster creation Creating a cluster that uses the external cloud provider and external azurediskcsi driver [OPTIONAL] with a 1 control plane nodes and 2 worker nodes
capz-e2e Workload cluster creation Creating a dual-stack cluster [OPTIONAL] With dual-stack worker node
capz-e2e Workload cluster creation Creating a highly available cluster [REQUIRED] With 3 control-plane nodes and 2 Linux and 2 Windows worker nodes
capz-e2e Workload cluster creation Creating a ipv6 control-plane cluster [REQUIRED] With ipv6 worker node
capz-e2e Workload cluster creation Creating a private cluster [OPTIONAL] Creates a public management cluster in a custom vnet
capz-e2e Workload cluster creation Creating an AKS cluster [EXPERIMENTAL][Managed Kubernetes] with a single control plane node and 1 node
capz-e2e Workload cluster creation Creating clusters using clusterclass [OPTIONAL] with a single control plane node, one linux worker node, and one windows worker node
capz-e2e [K8s-Upgrade] Running the workload cluster upgrade tests [CSI Migration] Running CSI migration test CSI=external CCM=external AzureDiskCSIMigration=true: upgrade to v1.23 should create volumes dynamically with intree cloud provider
capz-e2e [K8s-Upgrade] Running the workload cluster upgrade tests [CSI Migration] Running CSI migration test CSI=external CCM=internal AzureDiskCSIMigration=true: upgrade to v1.23 should create volumes dynamically with intree cloud provider
capz-e2e [K8s-Upgrade] Running the workload cluster upgrade tests [CSI Migration] Running CSI migration test CSI=internal CCM=internal AzureDiskCSIMigration=false: upgrade to v1.23 should create volumes dynamically with intree cloud provider
... skipping 591 lines ... [1mSTEP[0m: Creating log watcher for controller kube-system/coredns-bd6b6df9f-mff7d, container coredns [1mSTEP[0m: Fetching kube-system pod logs took 816.352642ms [1mSTEP[0m: Dumping workload cluster mhc-remediation-b9psf8/mhc-remediation-zlafbf Azure activity log [1mSTEP[0m: Creating log watcher for controller kube-system/calico-kube-controllers-85f479877b-z87b7, container calico-kube-controllers [1mSTEP[0m: Collecting events for Pod kube-system/kube-apiserver-mhc-remediation-zlafbf-control-plane-k7xbr [1mSTEP[0m: Collecting events for Pod kube-system/coredns-bd6b6df9f-zrjnm [1mSTEP[0m: failed to find events of Pod "kube-apiserver-mhc-remediation-zlafbf-control-plane-k7xbr" [1mSTEP[0m: Creating log watcher for controller kube-system/etcd-mhc-remediation-zlafbf-control-plane-k7xbr, container etcd [1mSTEP[0m: Collecting events for Pod kube-system/coredns-bd6b6df9f-mff7d [1mSTEP[0m: Creating log watcher for controller kube-system/coredns-bd6b6df9f-zrjnm, container coredns [1mSTEP[0m: Collecting events for Pod kube-system/kube-scheduler-mhc-remediation-zlafbf-control-plane-k7xbr [1mSTEP[0m: failed to find events of Pod "kube-scheduler-mhc-remediation-zlafbf-control-plane-k7xbr" [1mSTEP[0m: Creating log watcher for controller kube-system/kube-controller-manager-mhc-remediation-zlafbf-control-plane-k7xbr, container kube-controller-manager [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-ckvwg, container calico-node [1mSTEP[0m: Collecting events for Pod kube-system/calico-node-ckvwg [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-zttbp, container calico-node [1mSTEP[0m: Collecting events for Pod kube-system/etcd-mhc-remediation-zlafbf-control-plane-k7xbr [1mSTEP[0m: failed to find events of Pod "etcd-mhc-remediation-zlafbf-control-plane-k7xbr" [1mSTEP[0m: Collecting events for Pod kube-system/kube-proxy-4v96v [1mSTEP[0m: Creating log watcher for controller kube-system/kube-scheduler-mhc-remediation-zlafbf-control-plane-k7xbr, container kube-scheduler [1mSTEP[0m: Creating log watcher for controller kube-system/kube-apiserver-mhc-remediation-zlafbf-control-plane-k7xbr, container kube-apiserver [1mSTEP[0m: Collecting events for Pod kube-system/kube-controller-manager-mhc-remediation-zlafbf-control-plane-k7xbr [1mSTEP[0m: failed to find events of Pod "kube-controller-manager-mhc-remediation-zlafbf-control-plane-k7xbr" [1mSTEP[0m: Creating log watcher for controller kube-system/kube-proxy-4v96v, container kube-proxy [1mSTEP[0m: Creating log watcher for controller kube-system/kube-proxy-j9xlt, container kube-proxy [1mSTEP[0m: Collecting events for Pod kube-system/kube-proxy-j9xlt [1mSTEP[0m: Collecting events for Pod kube-system/calico-node-zttbp [1mSTEP[0m: Fetching activity logs took 1.833725147s [1mSTEP[0m: Dumping all the Cluster API resources in the "mhc-remediation-b9psf8" namespace ... skipping 20 lines ... [1mShould pivot the bootstrap cluster to a self-hosted cluster[0m [37m/home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/test/e2e/azure_selfhosted.go:107[0m INFO: "Should pivot the bootstrap cluster to a self-hosted cluster" started at Mon, 16 Jan 2023 17:12:02 UTC on Ginkgo node 10 of 10 [1mSTEP[0m: Creating namespace "self-hosted" for hosting the cluster Jan 16 17:12:02.659: INFO: starting to create namespace for hosting the "self-hosted" test spec 2023/01/16 17:12:02 failed trying to get namespace (self-hosted):namespaces "self-hosted" not found INFO: Creating namespace self-hosted INFO: Creating event watcher for namespace "self-hosted" [1mSTEP[0m: Creating a workload cluster INFO: Creating the workload cluster with name "self-hosted-4t36gw" using the "management" template (Kubernetes v1.23.15, 1 control-plane machines, 1 worker machines) INFO: Getting the cluster template yaml INFO: clusterctl config cluster self-hosted-4t36gw --infrastructure (default) --kubernetes-version v1.23.15 --control-plane-machine-count 1 --worker-machine-count 1 --flavor management ... skipping 62 lines ... [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-9wqnp, container calico-node [1mSTEP[0m: Creating log watcher for controller kube-system/kube-scheduler-self-hosted-4t36gw-control-plane-l2mbn, container kube-scheduler [1mSTEP[0m: Collecting events for Pod kube-system/coredns-bd6b6df9f-ztv7l [1mSTEP[0m: Fetching kube-system pod logs took 713.818176ms [1mSTEP[0m: Dumping workload cluster self-hosted/self-hosted-4t36gw Azure activity log [1mSTEP[0m: Collecting events for Pod kube-system/kube-scheduler-self-hosted-4t36gw-control-plane-l2mbn [1mSTEP[0m: failed to find events of Pod "kube-scheduler-self-hosted-4t36gw-control-plane-l2mbn" [1mSTEP[0m: Collecting events for Pod kube-system/calico-node-9wqnp [1mSTEP[0m: Creating log watcher for controller kube-system/calico-kube-controllers-85f479877b-rz7n6, container calico-kube-controllers [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-qlghn, container calico-node [1mSTEP[0m: Collecting events for Pod kube-system/kube-proxy-6ljvh [1mSTEP[0m: Creating log watcher for controller kube-system/kube-proxy-6ljvh, container kube-proxy [1mSTEP[0m: Creating log watcher for controller kube-system/kube-apiserver-self-hosted-4t36gw-control-plane-l2mbn, container kube-apiserver [1mSTEP[0m: Creating log watcher for controller kube-system/kube-proxy-v8965, container kube-proxy [1mSTEP[0m: Collecting events for Pod kube-system/kube-apiserver-self-hosted-4t36gw-control-plane-l2mbn [1mSTEP[0m: Collecting events for Pod kube-system/kube-proxy-v8965 [1mSTEP[0m: failed to find events of Pod "kube-apiserver-self-hosted-4t36gw-control-plane-l2mbn" [1mSTEP[0m: Collecting events for Pod kube-system/calico-node-qlghn [1mSTEP[0m: Creating log watcher for controller kube-system/kube-controller-manager-self-hosted-4t36gw-control-plane-l2mbn, container kube-controller-manager [1mSTEP[0m: Creating log watcher for controller kube-system/coredns-bd6b6df9f-gkhjl, container coredns [1mSTEP[0m: Collecting events for Pod kube-system/coredns-bd6b6df9f-gkhjl [1mSTEP[0m: Collecting events for Pod kube-system/kube-controller-manager-self-hosted-4t36gw-control-plane-l2mbn [1mSTEP[0m: failed to find events of Pod "kube-controller-manager-self-hosted-4t36gw-control-plane-l2mbn" [1mSTEP[0m: Creating log watcher for controller kube-system/coredns-bd6b6df9f-ztv7l, container coredns [1mSTEP[0m: Creating log watcher for controller kube-system/etcd-self-hosted-4t36gw-control-plane-l2mbn, container etcd [1mSTEP[0m: Collecting events for Pod kube-system/etcd-self-hosted-4t36gw-control-plane-l2mbn [1mSTEP[0m: failed to find events of Pod "etcd-self-hosted-4t36gw-control-plane-l2mbn" [1mSTEP[0m: Collecting events for Pod kube-system/calico-kube-controllers-85f479877b-rz7n6 [1mSTEP[0m: Fetching activity logs took 4.38993047s Jan 16 17:22:42.676: INFO: Dumping all the Cluster API resources in the "self-hosted" namespace Jan 16 17:22:43.352: INFO: Deleting all clusters in the self-hosted namespace [1mSTEP[0m: Deleting cluster self-hosted-4t36gw INFO: Waiting for the Cluster self-hosted/self-hosted-4t36gw to be deleted ... skipping 217 lines ... [1mSTEP[0m: Fetching kube-system pod logs took 1.069843127s [1mSTEP[0m: Collecting events for Pod kube-system/calico-node-l5gqn [1mSTEP[0m: Collecting events for Pod kube-system/kube-proxy-windows-5wqsq [1mSTEP[0m: Collecting events for Pod kube-system/kube-apiserver-machine-pool-jer8cv-control-plane-2sfp9 [1mSTEP[0m: Collecting events for Pod kube-system/etcd-machine-pool-jer8cv-control-plane-2sfp9 [1mSTEP[0m: Creating log watcher for controller kube-system/etcd-machine-pool-jer8cv-control-plane-2sfp9, container etcd [1mSTEP[0m: failed to find events of Pod "kube-apiserver-machine-pool-jer8cv-control-plane-2sfp9" [1mSTEP[0m: Creating log watcher for controller kube-system/kube-apiserver-machine-pool-jer8cv-control-plane-2sfp9, container kube-apiserver [1mSTEP[0m: Dumping workload cluster machine-pool-ri3wmj/machine-pool-jer8cv Azure activity log [1mSTEP[0m: failed to find events of Pod "etcd-machine-pool-jer8cv-control-plane-2sfp9" [1mSTEP[0m: Creating log watcher for controller kube-system/kube-scheduler-machine-pool-jer8cv-control-plane-2sfp9, container kube-scheduler [1mSTEP[0m: Collecting events for Pod kube-system/kube-scheduler-machine-pool-jer8cv-control-plane-2sfp9 [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-windows-2ktdz, container calico-node-startup [1mSTEP[0m: failed to find events of Pod "kube-scheduler-machine-pool-jer8cv-control-plane-2sfp9" [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-r669r, container calico-node [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-windows-2ktdz, container calico-node-felix [1mSTEP[0m: Creating log watcher for controller kube-system/coredns-bd6b6df9f-cllg7, container coredns [1mSTEP[0m: Creating log watcher for controller kube-system/kube-controller-manager-machine-pool-jer8cv-control-plane-2sfp9, container kube-controller-manager [1mSTEP[0m: Collecting events for Pod kube-system/kube-controller-manager-machine-pool-jer8cv-control-plane-2sfp9 [1mSTEP[0m: failed to find events of Pod "kube-controller-manager-machine-pool-jer8cv-control-plane-2sfp9" [1mSTEP[0m: Collecting events for Pod kube-system/coredns-bd6b6df9f-cllg7 [1mSTEP[0m: Creating log watcher for controller kube-system/kube-proxy-4n8mc, container kube-proxy [1mSTEP[0m: Collecting events for Pod kube-system/kube-proxy-4n8mc [1mSTEP[0m: Creating log watcher for controller kube-system/coredns-bd6b6df9f-pgqcq, container coredns [1mSTEP[0m: Creating log watcher for controller kube-system/kube-proxy-ml68v, container kube-proxy [1mSTEP[0m: Collecting events for Pod kube-system/kube-proxy-ml68v [1mSTEP[0m: Collecting events for Pod kube-system/coredns-bd6b6df9f-pgqcq [1mSTEP[0m: Creating log watcher for controller kube-system/kube-proxy-windows-5wqsq, container kube-proxy [1mSTEP[0m: Collecting events for Pod kube-system/calico-node-windows-2ktdz [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-l5gqn, container calico-node [1mSTEP[0m: Collecting events for Pod kube-system/calico-node-r669r [1mSTEP[0m: Collecting events for Pod kube-system/calico-kube-controllers-85f479877b-bkgtn [1mSTEP[0m: Creating log watcher for controller kube-system/calico-kube-controllers-85f479877b-bkgtn, container calico-kube-controllers [1mSTEP[0m: Error starting logs stream for pod kube-system/kube-proxy-windows-5wqsq, container kube-proxy: pods "win-p-win000002" not found [1mSTEP[0m: Error starting logs stream for pod kube-system/kube-proxy-ml68v, container kube-proxy: pods "machine-pool-jer8cv-mp-0000002" not found [1mSTEP[0m: Error starting logs stream for pod kube-system/calico-node-windows-2ktdz, container calico-node-startup: pods "win-p-win000002" not found [1mSTEP[0m: Error starting logs stream for pod kube-system/calico-node-windows-2ktdz, container calico-node-felix: pods "win-p-win000002" not found [1mSTEP[0m: Error starting logs stream for pod kube-system/calico-node-r669r, container calico-node: pods "machine-pool-jer8cv-mp-0000002" not found [1mSTEP[0m: Fetching activity logs took 1.403822833s [1mSTEP[0m: Dumping all the Cluster API resources in the "machine-pool-ri3wmj" namespace [1mSTEP[0m: Deleting cluster machine-pool-ri3wmj/machine-pool-jer8cv [1mSTEP[0m: Deleting cluster machine-pool-jer8cv INFO: Waiting for the Cluster machine-pool-ri3wmj/machine-pool-jer8cv to be deleted [1mSTEP[0m: Waiting for cluster machine-pool-jer8cv to be deleted ... skipping 72 lines ... Jan 16 17:20:35.458: INFO: Collecting logs for Windows node quick-sta-g6k5n in cluster quick-start-iw3jxc in namespace quick-start-ssv87a Jan 16 17:23:10.894: INFO: Attempting to copy file /c:/crashdumps.tar on node quick-sta-g6k5n to /logs/artifacts/clusters/quick-start-iw3jxc/machines/quick-start-iw3jxc-md-win-54547799f7-g9p4b/crashdumps.tar Jan 16 17:23:14.610: INFO: Collecting boot logs for AzureMachine quick-start-iw3jxc-md-win-g6k5n Failed to get logs for machine quick-start-iw3jxc-md-win-54547799f7-g9p4b, cluster quick-start-ssv87a/quick-start-iw3jxc: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1] Jan 16 17:23:15.870: INFO: Collecting logs for Windows node quick-sta-bzvzw in cluster quick-start-iw3jxc in namespace quick-start-ssv87a Jan 16 17:25:54.750: INFO: Attempting to copy file /c:/crashdumps.tar on node quick-sta-bzvzw to /logs/artifacts/clusters/quick-start-iw3jxc/machines/quick-start-iw3jxc-md-win-54547799f7-n8htl/crashdumps.tar Jan 16 17:25:58.206: INFO: Collecting boot logs for AzureMachine quick-start-iw3jxc-md-win-bzvzw Failed to get logs for machine quick-start-iw3jxc-md-win-54547799f7-n8htl, cluster quick-start-ssv87a/quick-start-iw3jxc: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1] [1mSTEP[0m: Dumping workload cluster quick-start-ssv87a/quick-start-iw3jxc kube-system pod logs [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-windows-bg9db, container calico-node-startup [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-windows-zrnfn, container calico-node-startup [1mSTEP[0m: Collecting events for Pod kube-system/csi-proxy-qp4lq [1mSTEP[0m: Collecting events for Pod kube-system/kube-proxy-5q9x8 [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-windows-zrnfn, container calico-node-felix ... skipping 5 lines ... [1mSTEP[0m: Collecting events for Pod kube-system/calico-node-windows-zrnfn [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-xrbqk, container calico-node [1mSTEP[0m: Collecting events for Pod kube-system/calico-node-lfhr8 [1mSTEP[0m: Fetching kube-system pod logs took 1.150373294s [1mSTEP[0m: Dumping workload cluster quick-start-ssv87a/quick-start-iw3jxc Azure activity log [1mSTEP[0m: Collecting events for Pod kube-system/etcd-quick-start-iw3jxc-control-plane-8w98b [1mSTEP[0m: failed to find events of Pod "etcd-quick-start-iw3jxc-control-plane-8w98b" [1mSTEP[0m: Creating log watcher for controller kube-system/kube-apiserver-quick-start-iw3jxc-control-plane-8w98b, container kube-apiserver [1mSTEP[0m: Creating log watcher for controller kube-system/coredns-bd6b6df9f-8dnrf, container coredns [1mSTEP[0m: Collecting events for Pod kube-system/kube-scheduler-quick-start-iw3jxc-control-plane-8w98b [1mSTEP[0m: Collecting events for Pod kube-system/kube-proxy-windows-hjjqr [1mSTEP[0m: failed to find events of Pod "kube-scheduler-quick-start-iw3jxc-control-plane-8w98b" [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-windows-bg9db, container calico-node-felix [1mSTEP[0m: Creating log watcher for controller kube-system/kube-proxy-x6njg, container kube-proxy [1mSTEP[0m: Collecting events for Pod kube-system/coredns-bd6b6df9f-8dnrf [1mSTEP[0m: Creating log watcher for controller kube-system/coredns-bd6b6df9f-dmf7w, container coredns [1mSTEP[0m: Collecting events for Pod kube-system/kube-proxy-x6njg [1mSTEP[0m: Collecting events for Pod kube-system/coredns-bd6b6df9f-dmf7w [1mSTEP[0m: Creating log watcher for controller kube-system/containerd-logger-bdxmd, container containerd-logger [1mSTEP[0m: Collecting events for Pod kube-system/kube-apiserver-quick-start-iw3jxc-control-plane-8w98b [1mSTEP[0m: Collecting events for Pod kube-system/calico-node-xrbqk [1mSTEP[0m: Creating log watcher for controller kube-system/kube-scheduler-quick-start-iw3jxc-control-plane-8w98b, container kube-scheduler [1mSTEP[0m: Creating log watcher for controller kube-system/csi-proxy-h5m4r, container csi-proxy [1mSTEP[0m: Collecting events for Pod kube-system/calico-node-windows-bg9db [1mSTEP[0m: Creating log watcher for controller kube-system/csi-proxy-qp4lq, container csi-proxy [1mSTEP[0m: failed to find events of Pod "kube-apiserver-quick-start-iw3jxc-control-plane-8w98b" [1mSTEP[0m: Collecting events for Pod kube-system/containerd-logger-bdxmd [1mSTEP[0m: Creating log watcher for controller kube-system/containerd-logger-k9lhm, container containerd-logger [1mSTEP[0m: Collecting events for Pod kube-system/containerd-logger-k9lhm [1mSTEP[0m: Collecting events for Pod kube-system/kube-controller-manager-quick-start-iw3jxc-control-plane-8w98b [1mSTEP[0m: failed to find events of Pod "kube-controller-manager-quick-start-iw3jxc-control-plane-8w98b" [1mSTEP[0m: Creating log watcher for controller kube-system/calico-kube-controllers-85f479877b-wnfs2, container calico-kube-controllers [1mSTEP[0m: Creating log watcher for controller kube-system/kube-controller-manager-quick-start-iw3jxc-control-plane-8w98b, container kube-controller-manager [1mSTEP[0m: Collecting events for Pod kube-system/csi-proxy-h5m4r [1mSTEP[0m: Creating log watcher for controller kube-system/kube-proxy-5q9x8, container kube-proxy [1mSTEP[0m: Collecting events for Pod kube-system/calico-kube-controllers-85f479877b-wnfs2 [1mSTEP[0m: Fetching activity logs took 1.485538962s ... skipping 105 lines ... [1mSTEP[0m: Collecting events for Pod kube-system/kube-proxy-bjwp5 [1mSTEP[0m: Creating log watcher for controller kube-system/kube-proxy-bptck, container kube-proxy [1mSTEP[0m: Collecting events for Pod kube-system/kube-scheduler-node-drain-ufkqp6-control-plane-grj6m [1mSTEP[0m: Creating log watcher for controller kube-system/kube-scheduler-node-drain-ufkqp6-control-plane-grj6m, container kube-scheduler [1mSTEP[0m: Creating log watcher for controller kube-system/kube-scheduler-node-drain-ufkqp6-control-plane-wpd9t, container kube-scheduler [1mSTEP[0m: Collecting events for Pod kube-system/kube-scheduler-node-drain-ufkqp6-control-plane-wpd9t [1mSTEP[0m: Error starting logs stream for pod kube-system/etcd-node-drain-ufkqp6-control-plane-grj6m, container etcd: pods "node-drain-ufkqp6-control-plane-grj6m" not found [1mSTEP[0m: Error starting logs stream for pod kube-system/kube-scheduler-node-drain-ufkqp6-control-plane-grj6m, container kube-scheduler: pods "node-drain-ufkqp6-control-plane-grj6m" not found [1mSTEP[0m: Error starting logs stream for pod kube-system/calico-node-4b629, container calico-node: pods "node-drain-ufkqp6-control-plane-grj6m" not found [1mSTEP[0m: Error starting logs stream for pod kube-system/kube-controller-manager-node-drain-ufkqp6-control-plane-grj6m, container kube-controller-manager: pods "node-drain-ufkqp6-control-plane-grj6m" not found [1mSTEP[0m: Error starting logs stream for pod kube-system/kube-apiserver-node-drain-ufkqp6-control-plane-grj6m, container kube-apiserver: pods "node-drain-ufkqp6-control-plane-grj6m" not found [1mSTEP[0m: Error starting logs stream for pod kube-system/kube-proxy-bjwp5, container kube-proxy: pods "node-drain-ufkqp6-control-plane-grj6m" not found [1mSTEP[0m: Fetching activity logs took 3.432447005s [1mSTEP[0m: Dumping all the Cluster API resources in the "node-drain-1t3p0b" namespace [1mSTEP[0m: Deleting cluster node-drain-1t3p0b/node-drain-ufkqp6 [1mSTEP[0m: Deleting cluster node-drain-ufkqp6 INFO: Waiting for the Cluster node-drain-1t3p0b/node-drain-ufkqp6 to be deleted [1mSTEP[0m: Waiting for cluster node-drain-ufkqp6 to be deleted ... skipping 78 lines ... Jan 16 17:23:49.222: INFO: Collecting logs for Windows node md-scale-j7kzb in cluster md-scale-9832bl in namespace md-scale-unqatw Jan 16 17:26:25.062: INFO: Attempting to copy file /c:/crashdumps.tar on node md-scale-j7kzb to /logs/artifacts/clusters/md-scale-9832bl/machines/md-scale-9832bl-md-win-86d79b5599-897mr/crashdumps.tar Jan 16 17:26:28.383: INFO: Collecting boot logs for AzureMachine md-scale-9832bl-md-win-j7kzb Failed to get logs for machine md-scale-9832bl-md-win-86d79b5599-897mr, cluster md-scale-unqatw/md-scale-9832bl: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1] Jan 16 17:26:29.637: INFO: Collecting logs for Windows node md-scale-8rmqz in cluster md-scale-9832bl in namespace md-scale-unqatw Jan 16 17:29:08.410: INFO: Attempting to copy file /c:/crashdumps.tar on node md-scale-8rmqz to /logs/artifacts/clusters/md-scale-9832bl/machines/md-scale-9832bl-md-win-86d79b5599-8w7ht/crashdumps.tar Jan 16 17:29:11.445: INFO: Collecting boot logs for AzureMachine md-scale-9832bl-md-win-8rmqz Failed to get logs for machine md-scale-9832bl-md-win-86d79b5599-8w7ht, cluster md-scale-unqatw/md-scale-9832bl: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1] [1mSTEP[0m: Dumping workload cluster md-scale-unqatw/md-scale-9832bl kube-system pod logs [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-9zxxp, container calico-node [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-qvf6z, container calico-node [1mSTEP[0m: Collecting events for Pod kube-system/kube-proxy-gcr2j [1mSTEP[0m: Collecting events for Pod kube-system/kube-proxy-windows-cn8q4 [1mSTEP[0m: Collecting events for Pod kube-system/calico-node-windows-q8x67 [1mSTEP[0m: Collecting events for Pod kube-system/calico-node-qvf6z [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-windows-cdmlp, container calico-node-startup [1mSTEP[0m: Collecting events for Pod kube-system/calico-kube-controllers-85f479877b-thngk [1mSTEP[0m: Collecting events for Pod kube-system/kube-scheduler-md-scale-9832bl-control-plane-nxtf5 [1mSTEP[0m: failed to find events of Pod "kube-scheduler-md-scale-9832bl-control-plane-nxtf5" [1mSTEP[0m: Creating log watcher for controller kube-system/csi-proxy-9mz2v, container csi-proxy [1mSTEP[0m: Creating log watcher for controller kube-system/calico-kube-controllers-85f479877b-thngk, container calico-kube-controllers [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-windows-cdmlp, container calico-node-felix [1mSTEP[0m: Fetching kube-system pod logs took 1.115292992s [1mSTEP[0m: Dumping workload cluster md-scale-unqatw/md-scale-9832bl Azure activity log [1mSTEP[0m: Creating log watcher for controller kube-system/containerd-logger-7zwbg, container containerd-logger ... skipping 19 lines ... [1mSTEP[0m: Creating log watcher for controller kube-system/csi-proxy-vsxrf, container csi-proxy [1mSTEP[0m: Collecting events for Pod kube-system/coredns-bd6b6df9f-2v64q [1mSTEP[0m: Creating log watcher for controller kube-system/coredns-bd6b6df9f-jmb4h, container coredns [1mSTEP[0m: Creating log watcher for controller kube-system/calico-node-windows-q8x67, container calico-node-felix [1mSTEP[0m: Creating log watcher for controller kube-system/containerd-logger-wdbwz, container containerd-logger [1mSTEP[0m: Collecting events for Pod kube-system/kube-controller-manager-md-scale-9832bl-control-plane-nxtf5 [1mSTEP[0m: failed to find events of Pod "kube-controller-manager-md-scale-9832bl-control-plane-nxtf5" [1mSTEP[0m: failed to find events of Pod "etcd-md-scale-9832bl-control-plane-nxtf5" [1mSTEP[0m: failed to find events of Pod "kube-apiserver-md-scale-9832bl-control-plane-nxtf5" [1mSTEP[0m: Collecting events for Pod kube-system/containerd-logger-wdbwz [1mSTEP[0m: Collecting events for Pod kube-system/containerd-logger-7zwbg [1mSTEP[0m: Fetching activity logs took 3.967866721s [1mSTEP[0m: Dumping all the Cluster API resources in the "md-scale-unqatw" namespace [1mSTEP[0m: Deleting cluster md-scale-unqatw/md-scale-9832bl [1mSTEP[0m: Deleting cluster md-scale-9832bl ... skipping 11 lines ... [90m/home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/test/e2e/capi_test.go:45[0m Should successfully scale out and scale in a MachineDeployment [90m/home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/test/e2e/capi_test.go:171[0m Should successfully scale a MachineDeployment up and down upon changes to the MachineDeployment replica count [90m/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.2.9/e2e/md_scale.go:71[0m [90m------------------------------[0m {"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:164","func":"k8s.io/test-infra/prow/entrypoint.Options.ExecuteProcess","level":"error","msg":"Process did not finish before 4h0m0s timeout","severity":"error","time":"2023-01-16T20:59:50Z"} ++ early_exit_handler ++ '[' -n 155 ']' ++ kill -TERM 155 ++ cleanup_dind ++ [[ true == \t\r\u\e ]] ++ echo 'Cleaning up after docker' ... skipping 12 lines ... Cleaning up after docker Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? Stopping Docker: dockerProgram process in pidfile '/var/run/docker-ssd.pid', 1 process(es), refused to die. ================================================================================ Done cleaning up after docker in docker. All sensitive variables are redacted {"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:254","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Process did not exit before 15m0s grace period","severity":"error","time":"2023-01-16T21:14:51Z"} {"component":"entrypoint","error":"os: process already finished","file":"k8s.io/test-infra/prow/entrypoint/run.go:256","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Could not kill process after grace period","severity":"error","time":"2023-01-16T21:14:51Z"}