This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 0 failed / 7 succeeded
Started2022-09-23 20:36
Elapsed4h15m
Revisionrelease-1.5

No Test Failures!


Show 7 Passed Tests

Show 17 Skipped Tests

Error lines from build-log.txt

... skipping 741 lines ...
  Should pivot the bootstrap cluster to a self-hosted cluster
  /home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/test/e2e/azure_selfhosted.go:107

INFO: "Should pivot the bootstrap cluster to a self-hosted cluster" started at Fri, 23 Sep 2022 20:45:29 UTC on Ginkgo node 2 of 10
STEP: Creating namespace "self-hosted" for hosting the cluster
Sep 23 20:45:29.546: INFO: starting to create namespace for hosting the "self-hosted" test spec
2022/09/23 20:45:29 failed trying to get namespace (self-hosted):namespaces "self-hosted" not found
INFO: Creating namespace self-hosted
INFO: Creating event watcher for namespace "self-hosted"
STEP: Creating a workload cluster
INFO: Creating the workload cluster with name "self-hosted-35b5lk" using the "management" template (Kubernetes v1.23.12, 1 control-plane machines, 1 worker machines)
INFO: Getting the cluster template yaml
INFO: clusterctl config cluster self-hosted-35b5lk --infrastructure (default) --kubernetes-version v1.23.12 --control-plane-machine-count 1 --worker-machine-count 1 --flavor management
... skipping 66 lines ...
STEP: Creating log watcher for controller kube-system/calico-node-n6xw7, container calico-node
STEP: Collecting events for Pod kube-system/calico-node-n6xw7
STEP: Creating log watcher for controller kube-system/calico-node-pcmdk, container calico-node
STEP: Collecting events for Pod kube-system/kube-apiserver-self-hosted-35b5lk-control-plane-hqc6l
STEP: Creating log watcher for controller kube-system/kube-proxy-vlrmj, container kube-proxy
STEP: Creating log watcher for controller kube-system/kube-scheduler-self-hosted-35b5lk-control-plane-hqc6l, container kube-scheduler
STEP: failed to find events of Pod "kube-apiserver-self-hosted-35b5lk-control-plane-hqc6l"
STEP: Collecting events for Pod kube-system/calico-node-pcmdk
STEP: Collecting events for Pod kube-system/kube-scheduler-self-hosted-35b5lk-control-plane-hqc6l
STEP: Creating log watcher for controller kube-system/coredns-64897985d-52blz, container coredns
STEP: failed to find events of Pod "kube-scheduler-self-hosted-35b5lk-control-plane-hqc6l"
STEP: Collecting events for Pod kube-system/kube-controller-manager-self-hosted-35b5lk-control-plane-hqc6l
STEP: Creating log watcher for controller kube-system/kube-controller-manager-self-hosted-35b5lk-control-plane-hqc6l, container kube-controller-manager
STEP: Collecting events for Pod kube-system/kube-proxy-vlrmj
STEP: Collecting events for Pod kube-system/coredns-64897985d-s5scj
STEP: Collecting events for Pod kube-system/coredns-64897985d-52blz
STEP: Creating log watcher for controller kube-system/coredns-64897985d-s5scj, container coredns
STEP: Creating log watcher for controller kube-system/kube-proxy-ttl6t, container kube-proxy
STEP: Creating log watcher for controller kube-system/etcd-self-hosted-35b5lk-control-plane-hqc6l, container etcd
STEP: Collecting events for Pod kube-system/kube-proxy-ttl6t
STEP: Creating log watcher for controller kube-system/kube-apiserver-self-hosted-35b5lk-control-plane-hqc6l, container kube-apiserver
STEP: Collecting events for Pod kube-system/etcd-self-hosted-35b5lk-control-plane-hqc6l
STEP: failed to find events of Pod "etcd-self-hosted-35b5lk-control-plane-hqc6l"
STEP: Fetching activity logs took 1.694960886s
Sep 23 20:58:40.188: INFO: Dumping all the Cluster API resources in the "self-hosted" namespace
Sep 23 20:58:40.529: INFO: Deleting all clusters in the self-hosted namespace
STEP: Deleting cluster self-hosted-35b5lk
INFO: Waiting for the Cluster self-hosted/self-hosted-35b5lk to be deleted
STEP: Waiting for cluster self-hosted-35b5lk to be deleted
INFO: Got error while streaming logs for pod capi-kubeadm-control-plane-system/capi-kubeadm-control-plane-controller-manager-74b6b6b77f-p9p5l, container manager: http2: client connection lost
INFO: Got error while streaming logs for pod capi-system/capi-controller-manager-7df9bc44b4-hxlzt, container manager: http2: client connection lost
INFO: Got error while streaming logs for pod capi-kubeadm-bootstrap-system/capi-kubeadm-bootstrap-controller-manager-6c76c59d6b-4tn78, container manager: http2: client connection lost
INFO: Got error while streaming logs for pod capz-system/capz-controller-manager-64754b48cf-4pvtz, container manager: http2: client connection lost
Sep 23 21:04:20.742: INFO: Deleting namespace used for hosting the "self-hosted" test spec
INFO: Deleting namespace self-hosted
Sep 23 21:04:20.760: INFO: Checking if any resources are left over in Azure for spec "self-hosted"
STEP: Redacting sensitive information from logs
Sep 23 21:05:09.138: INFO: Cleaning up after "Running the Cluster API E2E tests Running the self-hosted spec Should pivot the bootstrap cluster to a self-hosted cluster" spec
STEP: Redacting sensitive information from logs
... skipping 89 lines ...
STEP: Collecting events for Pod kube-system/kube-proxy-windows-tsgwd
STEP: Collecting events for Pod kube-system/kube-proxy-pms47
STEP: Creating log watcher for controller kube-system/kube-proxy-windows-tsgwd, container kube-proxy
STEP: Creating log watcher for controller kube-system/kube-scheduler-machine-pool-n15kro-control-plane-zwxrl, container kube-scheduler
STEP: Collecting events for Pod kube-system/kube-controller-manager-machine-pool-n15kro-control-plane-zwxrl
STEP: Creating log watcher for controller kube-system/etcd-machine-pool-n15kro-control-plane-zwxrl, container etcd
STEP: failed to find events of Pod "kube-controller-manager-machine-pool-n15kro-control-plane-zwxrl"
STEP: Creating log watcher for controller kube-system/kube-proxy-9mvqz, container kube-proxy
STEP: Collecting events for Pod kube-system/kube-apiserver-machine-pool-n15kro-control-plane-zwxrl
STEP: failed to find events of Pod "etcd-machine-pool-n15kro-control-plane-zwxrl"
STEP: Creating log watcher for controller kube-system/coredns-64897985d-pf78x, container coredns
STEP: Creating log watcher for controller kube-system/kube-controller-manager-machine-pool-n15kro-control-plane-zwxrl, container kube-controller-manager
STEP: Collecting events for Pod kube-system/coredns-64897985d-pf78x
STEP: Creating log watcher for controller kube-system/calico-node-windows-7frn6, container calico-node-startup
STEP: Collecting events for Pod kube-system/kube-scheduler-machine-pool-n15kro-control-plane-zwxrl
STEP: failed to find events of Pod "kube-scheduler-machine-pool-n15kro-control-plane-zwxrl"
STEP: Creating log watcher for controller kube-system/calico-node-windows-7frn6, container calico-node-felix
STEP: Error starting logs stream for pod kube-system/calico-node-df4pk, container calico-node: pods "machine-pool-n15kro-mp-0000002" not found
STEP: Error starting logs stream for pod kube-system/calico-node-windows-7frn6, container calico-node-startup: pods "win-p-win000002" not found
STEP: Error starting logs stream for pod kube-system/kube-proxy-9mvqz, container kube-proxy: pods "machine-pool-n15kro-mp-0000002" not found
STEP: Error starting logs stream for pod kube-system/kube-proxy-windows-tsgwd, container kube-proxy: pods "win-p-win000002" not found
STEP: Error starting logs stream for pod kube-system/calico-node-windows-7frn6, container calico-node-felix: pods "win-p-win000002" not found
STEP: Fetching activity logs took 14.345343132s
STEP: Dumping all the Cluster API resources in the "machine-pool-g2lqf2" namespace
STEP: Deleting cluster machine-pool-g2lqf2/machine-pool-n15kro
STEP: Deleting cluster machine-pool-n15kro
INFO: Waiting for the Cluster machine-pool-g2lqf2/machine-pool-n15kro to be deleted
STEP: Waiting for cluster machine-pool-n15kro to be deleted
... skipping 72 lines ...

Sep 23 20:55:17.898: INFO: Collecting logs for Windows node quick-sta-sxs8c in cluster quick-start-ck18be in namespace quick-start-smu4yq

Sep 23 20:57:56.529: INFO: Attempting to copy file /c:/crashdumps.tar on node quick-sta-sxs8c to /logs/artifacts/clusters/quick-start-ck18be/machines/quick-start-ck18be-md-win-568cdd8686-422qt/crashdumps.tar
Sep 23 20:57:59.986: INFO: Collecting boot logs for AzureMachine quick-start-ck18be-md-win-sxs8c

Failed to get logs for machine quick-start-ck18be-md-win-568cdd8686-422qt, cluster quick-start-smu4yq/quick-start-ck18be: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1]
Sep 23 20:58:01.158: INFO: Collecting logs for Windows node quick-sta-wv6hk in cluster quick-start-ck18be in namespace quick-start-smu4yq

Sep 23 21:00:39.802: INFO: Attempting to copy file /c:/crashdumps.tar on node quick-sta-wv6hk to /logs/artifacts/clusters/quick-start-ck18be/machines/quick-start-ck18be-md-win-568cdd8686-4g9cq/crashdumps.tar
Sep 23 21:00:43.294: INFO: Collecting boot logs for AzureMachine quick-start-ck18be-md-win-wv6hk

Failed to get logs for machine quick-start-ck18be-md-win-568cdd8686-4g9cq, cluster quick-start-smu4yq/quick-start-ck18be: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1]
STEP: Dumping workload cluster quick-start-smu4yq/quick-start-ck18be kube-system pod logs
STEP: Creating log watcher for controller kube-system/calico-node-v87zg, container calico-node
STEP: Fetching kube-system pod logs took 1.134995501s
STEP: Dumping workload cluster quick-start-smu4yq/quick-start-ck18be Azure activity log
STEP: Creating log watcher for controller kube-system/calico-node-whq6p, container calico-node
STEP: Creating log watcher for controller kube-system/calico-node-windows-m4szg, container calico-node-felix
... skipping 31 lines ...
STEP: Collecting events for Pod kube-system/kube-proxy-qw2q4
STEP: Creating log watcher for controller kube-system/kube-proxy-windows-2wvqv, container kube-proxy
STEP: Collecting events for Pod kube-system/coredns-64897985d-j9hhm
STEP: Collecting events for Pod kube-system/kube-proxy-windows-2wvqv
STEP: Collecting events for Pod kube-system/kube-proxy-fdkpr
STEP: Creating log watcher for controller kube-system/kube-proxy-qw2q4, container kube-proxy
STEP: failed to find events of Pod "kube-scheduler-quick-start-ck18be-control-plane-hqhg4"
STEP: failed to find events of Pod "kube-controller-manager-quick-start-ck18be-control-plane-hqhg4"
STEP: failed to find events of Pod "etcd-quick-start-ck18be-control-plane-hqhg4"
STEP: Fetching activity logs took 3.250921591s
STEP: Dumping all the Cluster API resources in the "quick-start-smu4yq" namespace
STEP: Deleting cluster quick-start-smu4yq/quick-start-ck18be
STEP: Deleting cluster quick-start-ck18be
INFO: Waiting for the Cluster quick-start-smu4yq/quick-start-ck18be to be deleted
STEP: Waiting for cluster quick-start-ck18be to be deleted
... skipping 100 lines ...
STEP: Collecting events for Pod kube-system/kube-proxy-wv7ck
STEP: Collecting events for Pod kube-system/kube-apiserver-node-drain-gtxmb5-control-plane-6bxcl
STEP: Collecting events for Pod kube-system/calico-kube-controllers-85f479877b-kbc8q
STEP: Collecting events for Pod kube-system/kube-controller-manager-node-drain-gtxmb5-control-plane-rnjzf
STEP: Creating log watcher for controller kube-system/kube-proxy-ld6h5, container kube-proxy
STEP: Creating log watcher for controller kube-system/kube-apiserver-node-drain-gtxmb5-control-plane-rnjzf, container kube-apiserver
STEP: Error starting logs stream for pod kube-system/kube-proxy-wv7ck, container kube-proxy: pods "node-drain-gtxmb5-control-plane-rnjzf" not found
STEP: Error starting logs stream for pod kube-system/etcd-node-drain-gtxmb5-control-plane-rnjzf, container etcd: pods "node-drain-gtxmb5-control-plane-rnjzf" not found
STEP: Error starting logs stream for pod kube-system/kube-controller-manager-node-drain-gtxmb5-control-plane-rnjzf, container kube-controller-manager: pods "node-drain-gtxmb5-control-plane-rnjzf" not found
STEP: Error starting logs stream for pod kube-system/kube-apiserver-node-drain-gtxmb5-control-plane-rnjzf, container kube-apiserver: pods "node-drain-gtxmb5-control-plane-rnjzf" not found
STEP: Error starting logs stream for pod kube-system/calico-node-4cmw7, container calico-node: pods "node-drain-gtxmb5-control-plane-rnjzf" not found
STEP: Error starting logs stream for pod kube-system/kube-scheduler-node-drain-gtxmb5-control-plane-rnjzf, container kube-scheduler: pods "node-drain-gtxmb5-control-plane-rnjzf" not found
STEP: Fetching activity logs took 2.072713402s
STEP: Dumping all the Cluster API resources in the "node-drain-zswat6" namespace
STEP: Deleting cluster node-drain-zswat6/node-drain-gtxmb5
STEP: Deleting cluster node-drain-gtxmb5
INFO: Waiting for the Cluster node-drain-zswat6/node-drain-gtxmb5 to be deleted
STEP: Waiting for cluster node-drain-gtxmb5 to be deleted
... skipping 214 lines ...

Sep 23 20:58:27.391: INFO: Collecting logs for Windows node md-scale-ttpdl in cluster md-scale-h1hdlg in namespace md-scale-gwxcs5

Sep 23 21:01:02.917: INFO: Attempting to copy file /c:/crashdumps.tar on node md-scale-ttpdl to /logs/artifacts/clusters/md-scale-h1hdlg/machines/md-scale-h1hdlg-md-win-78fbb8977d-4vfwq/crashdumps.tar
Sep 23 21:01:06.270: INFO: Collecting boot logs for AzureMachine md-scale-h1hdlg-md-win-ttpdl

Failed to get logs for machine md-scale-h1hdlg-md-win-78fbb8977d-4vfwq, cluster md-scale-gwxcs5/md-scale-h1hdlg: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1]
Sep 23 21:01:07.447: INFO: Collecting logs for Windows node md-scale-crvtw in cluster md-scale-h1hdlg in namespace md-scale-gwxcs5

Sep 23 21:03:45.213: INFO: Attempting to copy file /c:/crashdumps.tar on node md-scale-crvtw to /logs/artifacts/clusters/md-scale-h1hdlg/machines/md-scale-h1hdlg-md-win-78fbb8977d-xrrxj/crashdumps.tar
Sep 23 21:03:48.818: INFO: Collecting boot logs for AzureMachine md-scale-h1hdlg-md-win-crvtw

Failed to get logs for machine md-scale-h1hdlg-md-win-78fbb8977d-xrrxj, cluster md-scale-gwxcs5/md-scale-h1hdlg: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1]
STEP: Dumping workload cluster md-scale-gwxcs5/md-scale-h1hdlg kube-system pod logs
STEP: Fetching kube-system pod logs took 1.163318373s
STEP: Creating log watcher for controller kube-system/calico-kube-controllers-85f479877b-5kw29, container calico-kube-controllers
STEP: Creating log watcher for controller kube-system/kube-proxy-8fd8q, container kube-proxy
STEP: Creating log watcher for controller kube-system/kube-controller-manager-md-scale-h1hdlg-control-plane-vbfqc, container kube-controller-manager
STEP: Collecting events for Pod kube-system/calico-node-windows-87thk
... skipping 31 lines ...
STEP: Creating log watcher for controller kube-system/csi-proxy-lsfhj, container csi-proxy
STEP: Collecting events for Pod kube-system/kube-proxy-windows-nblrf
STEP: Collecting events for Pod kube-system/kube-apiserver-md-scale-h1hdlg-control-plane-vbfqc
STEP: Collecting events for Pod kube-system/calico-node-lcvfs
STEP: Creating log watcher for controller kube-system/calico-node-windows-87thk, container calico-node-startup
STEP: Creating log watcher for controller kube-system/calico-node-windows-87thk, container calico-node-felix
STEP: failed to find events of Pod "etcd-md-scale-h1hdlg-control-plane-vbfqc"
STEP: failed to find events of Pod "kube-scheduler-md-scale-h1hdlg-control-plane-vbfqc"
STEP: failed to find events of Pod "kube-controller-manager-md-scale-h1hdlg-control-plane-vbfqc"
STEP: failed to find events of Pod "kube-apiserver-md-scale-h1hdlg-control-plane-vbfqc"
STEP: Fetching activity logs took 1.769923432s
STEP: Dumping all the Cluster API resources in the "md-scale-gwxcs5" namespace
STEP: Deleting cluster md-scale-gwxcs5/md-scale-h1hdlg
STEP: Deleting cluster md-scale-h1hdlg
INFO: Waiting for the Cluster md-scale-gwxcs5/md-scale-h1hdlg to be deleted
STEP: Waiting for cluster md-scale-h1hdlg to be deleted
... skipping 9 lines ...
/home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/test/e2e/capi_test.go:45
  Should successfully scale out and scale in a MachineDeployment
  /home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/test/e2e/capi_test.go:183
    Should successfully scale a MachineDeployment up and down upon changes to the MachineDeployment replica count
    /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.2.1/e2e/md_scale.go:71
------------------------------
{"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:165","func":"k8s.io/test-infra/prow/entrypoint.Options.ExecuteProcess","level":"error","msg":"Process did not finish before 4h0m0s timeout","severity":"error","time":"2022-09-24T00:36:10Z"}
++ early_exit_handler
++ '[' -n 165 ']'
++ kill -TERM 165
++ cleanup_dind
++ [[ true == \t\r\u\e ]]
++ echo 'Cleaning up after docker'
... skipping 12 lines ...
Cleaning up after docker
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Stopping Docker: dockerProgram process in pidfile '/var/run/docker-ssd.pid', 1 process(es), refused to die.
================================================================================
Done cleaning up after docker in docker.
All sensitive variables are redacted
{"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:255","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Process did not exit before 15m0s grace period","severity":"error","time":"2022-09-24T00:51:10Z"}
{"component":"entrypoint","error":"os: process already finished","file":"k8s.io/test-infra/prow/entrypoint/run.go:257","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Could not kill process after grace period","severity":"error","time":"2022-09-24T00:51:10Z"}