This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 0 failed / 7 succeeded
Started2022-09-30 20:38
Elapsed4h15m
Revisionrelease-1.5

No Test Failures!


Show 7 Passed Tests

Show 14 Skipped Tests

Error lines from build-log.txt

... skipping 603 lines ...
STEP: Creating log watcher for controller kube-system/coredns-64897985d-42vcj, container coredns
STEP: Dumping workload cluster kcp-adoption-7eg74d/kcp-adoption-0cwun3 Azure activity log
STEP: Collecting events for Pod kube-system/calico-node-k9kws
STEP: Creating log watcher for controller kube-system/coredns-64897985d-887ph, container coredns
STEP: Collecting events for Pod kube-system/coredns-64897985d-887ph
STEP: Collecting events for Pod kube-system/kube-controller-manager-kcp-adoption-0cwun3-control-plane-0
STEP: failed to find events of Pod "etcd-kcp-adoption-0cwun3-control-plane-0"
STEP: Collecting events for Pod kube-system/kube-scheduler-kcp-adoption-0cwun3-control-plane-0
STEP: failed to find events of Pod "kube-scheduler-kcp-adoption-0cwun3-control-plane-0"
STEP: Creating log watcher for controller kube-system/kube-apiserver-kcp-adoption-0cwun3-control-plane-0, container kube-apiserver
STEP: Collecting events for Pod kube-system/kube-proxy-8llxr
STEP: Creating log watcher for controller kube-system/etcd-kcp-adoption-0cwun3-control-plane-0, container etcd
STEP: failed to find events of Pod "kube-controller-manager-kcp-adoption-0cwun3-control-plane-0"
STEP: Creating log watcher for controller kube-system/kube-proxy-8llxr, container kube-proxy
STEP: failed to find events of Pod "kube-apiserver-kcp-adoption-0cwun3-control-plane-0"
STEP: Fetching activity logs took 3.276714229s
STEP: Dumping all the Cluster API resources in the "kcp-adoption-7eg74d" namespace
STEP: Deleting cluster kcp-adoption-7eg74d/kcp-adoption-0cwun3
STEP: Deleting cluster kcp-adoption-0cwun3
INFO: Waiting for the Cluster kcp-adoption-7eg74d/kcp-adoption-0cwun3 to be deleted
STEP: Waiting for cluster kcp-adoption-0cwun3 to be deleted
... skipping 118 lines ...
  Should pivot the bootstrap cluster to a self-hosted cluster
  /home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/test/e2e/azure_selfhosted.go:107

INFO: "Should pivot the bootstrap cluster to a self-hosted cluster" started at Fri, 30 Sep 2022 20:47:24 UTC on Ginkgo node 9 of 10
STEP: Creating namespace "self-hosted" for hosting the cluster
Sep 30 20:47:24.112: INFO: starting to create namespace for hosting the "self-hosted" test spec
2022/09/30 20:47:24 failed trying to get namespace (self-hosted):namespaces "self-hosted" not found
INFO: Creating namespace self-hosted
INFO: Creating event watcher for namespace "self-hosted"
STEP: Creating a workload cluster
INFO: Creating the workload cluster with name "self-hosted-dix2kt" using the "management" template (Kubernetes v1.23.12, 1 control-plane machines, 1 worker machines)
INFO: Getting the cluster template yaml
INFO: clusterctl config cluster self-hosted-dix2kt --infrastructure (default) --kubernetes-version v1.23.12 --control-plane-machine-count 1 --worker-machine-count 1 --flavor management
... skipping 68 lines ...
STEP: Collecting events for Pod kube-system/kube-proxy-dts8q
STEP: Creating log watcher for controller kube-system/coredns-64897985d-qgkqb, container coredns
STEP: Collecting events for Pod kube-system/calico-node-92vs6
STEP: Creating log watcher for controller kube-system/calico-kube-controllers-85f479877b-7tv89, container calico-kube-controllers
STEP: Collecting events for Pod kube-system/etcd-self-hosted-dix2kt-control-plane-dzs59
STEP: Creating log watcher for controller kube-system/kube-proxy-dr75q, container kube-proxy
STEP: failed to find events of Pod "etcd-self-hosted-dix2kt-control-plane-dzs59"
STEP: Collecting events for Pod kube-system/kube-proxy-dr75q
STEP: Dumping workload cluster self-hosted/self-hosted-dix2kt Azure activity log
STEP: Creating log watcher for controller kube-system/calico-node-v9xnv, container calico-node
STEP: Creating log watcher for controller kube-system/calico-node-92vs6, container calico-node
STEP: Collecting events for Pod kube-system/calico-kube-controllers-85f479877b-7tv89
STEP: Collecting events for Pod kube-system/kube-apiserver-self-hosted-dix2kt-control-plane-dzs59
... skipping 6 lines ...
STEP: Fetching activity logs took 1.627079964s
Sep 30 20:56:32.690: INFO: Dumping all the Cluster API resources in the "self-hosted" namespace
Sep 30 20:56:33.219: INFO: Deleting all clusters in the self-hosted namespace
STEP: Deleting cluster self-hosted-dix2kt
INFO: Waiting for the Cluster self-hosted/self-hosted-dix2kt to be deleted
STEP: Waiting for cluster self-hosted-dix2kt to be deleted
INFO: Got error while streaming logs for pod capi-kubeadm-bootstrap-system/capi-kubeadm-bootstrap-controller-manager-6c76c59d6b-8xn4d, container manager: http2: client connection lost
INFO: Got error while streaming logs for pod capz-system/capz-controller-manager-8676764c86-rmkg9, container manager: http2: client connection lost
Sep 30 21:01:13.441: INFO: Deleting namespace used for hosting the "self-hosted" test spec
INFO: Deleting namespace self-hosted
Sep 30 21:01:13.471: INFO: Checking if any resources are left over in Azure for spec "self-hosted"
STEP: Redacting sensitive information from logs
Sep 30 21:01:42.381: INFO: Cleaning up after "Running the Cluster API E2E tests Running the self-hosted spec Should pivot the bootstrap cluster to a self-hosted cluster" spec
STEP: Redacting sensitive information from logs
... skipping 210 lines ...

Sep 30 20:59:16.665: INFO: Collecting logs for Windows node md-scale-42rvp in cluster md-scale-yzpnko in namespace md-scale-tcemkb

Sep 30 21:01:46.100: INFO: Attempting to copy file /c:/crashdumps.tar on node md-scale-42rvp to /logs/artifacts/clusters/md-scale-yzpnko/machines/md-scale-yzpnko-md-win-6ff6dffd9c-g9f44/crashdumps.tar
Sep 30 21:01:47.825: INFO: Collecting boot logs for AzureMachine md-scale-yzpnko-md-win-42rvp

Failed to get logs for machine md-scale-yzpnko-md-win-6ff6dffd9c-g9f44, cluster md-scale-tcemkb/md-scale-yzpnko: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1]
Sep 30 21:01:48.631: INFO: Collecting logs for Windows node md-scale-wmqqd in cluster md-scale-yzpnko in namespace md-scale-tcemkb

Sep 30 21:04:22.490: INFO: Attempting to copy file /c:/crashdumps.tar on node md-scale-wmqqd to /logs/artifacts/clusters/md-scale-yzpnko/machines/md-scale-yzpnko-md-win-6ff6dffd9c-shkbg/crashdumps.tar
Sep 30 21:04:24.278: INFO: Collecting boot logs for AzureMachine md-scale-yzpnko-md-win-wmqqd

Failed to get logs for machine md-scale-yzpnko-md-win-6ff6dffd9c-shkbg, cluster md-scale-tcemkb/md-scale-yzpnko: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1]
STEP: Dumping workload cluster md-scale-tcemkb/md-scale-yzpnko kube-system pod logs
STEP: Fetching kube-system pod logs took 433.923674ms
STEP: Collecting events for Pod kube-system/coredns-64897985d-pzc7p
STEP: Collecting events for Pod kube-system/csi-proxy-ddf8w
STEP: Creating log watcher for controller kube-system/calico-node-windows-bc6r2, container calico-node-startup
STEP: Collecting events for Pod kube-system/kube-proxy-8464x
... skipping 8 lines ...
STEP: Collecting events for Pod kube-system/kube-proxy-d94g2
STEP: Creating log watcher for controller kube-system/kube-proxy-windows-fx6nc, container kube-proxy
STEP: Creating log watcher for controller kube-system/kube-apiserver-md-scale-yzpnko-control-plane-hcxhq, container kube-apiserver
STEP: Collecting events for Pod kube-system/kube-proxy-windows-fx6nc
STEP: Collecting events for Pod kube-system/calico-node-c9phc
STEP: Collecting events for Pod kube-system/kube-apiserver-md-scale-yzpnko-control-plane-hcxhq
STEP: failed to find events of Pod "kube-apiserver-md-scale-yzpnko-control-plane-hcxhq"
STEP: Collecting events for Pod kube-system/kube-controller-manager-md-scale-yzpnko-control-plane-hcxhq
STEP: Creating log watcher for controller kube-system/calico-node-nmc9d, container calico-node
STEP: failed to find events of Pod "kube-controller-manager-md-scale-yzpnko-control-plane-hcxhq"
STEP: Creating log watcher for controller kube-system/calico-node-windows-52m7r, container calico-node-felix
STEP: Collecting events for Pod kube-system/calico-node-windows-52m7r
STEP: Creating log watcher for controller kube-system/kube-proxy-windows-6cw64, container kube-proxy
STEP: Collecting events for Pod kube-system/calico-node-windows-bc6r2
STEP: Creating log watcher for controller kube-system/containerd-logger-5vfmm, container containerd-logger
STEP: Collecting events for Pod kube-system/kube-proxy-windows-6cw64
STEP: Creating log watcher for controller kube-system/kube-proxy-8464x, container kube-proxy
STEP: Collecting events for Pod kube-system/etcd-md-scale-yzpnko-control-plane-hcxhq
STEP: Creating log watcher for controller kube-system/kube-scheduler-md-scale-yzpnko-control-plane-hcxhq, container kube-scheduler
STEP: failed to find events of Pod "etcd-md-scale-yzpnko-control-plane-hcxhq"
STEP: Creating log watcher for controller kube-system/csi-proxy-59rwl, container csi-proxy
STEP: Creating log watcher for controller kube-system/coredns-64897985d-mtb49, container coredns
STEP: Creating log watcher for controller kube-system/containerd-logger-vzh2w, container containerd-logger
STEP: Creating log watcher for controller kube-system/coredns-64897985d-pzc7p, container coredns
STEP: Collecting events for Pod kube-system/coredns-64897985d-mtb49
STEP: Collecting events for Pod kube-system/csi-proxy-59rwl
STEP: Collecting events for Pod kube-system/containerd-logger-vzh2w
STEP: Creating log watcher for controller kube-system/calico-kube-controllers-85f479877b-cbhpj, container calico-kube-controllers
STEP: Collecting events for Pod kube-system/kube-scheduler-md-scale-yzpnko-control-plane-hcxhq
STEP: Creating log watcher for controller kube-system/csi-proxy-ddf8w, container csi-proxy
STEP: Creating log watcher for controller kube-system/kube-controller-manager-md-scale-yzpnko-control-plane-hcxhq, container kube-controller-manager
STEP: Collecting events for Pod kube-system/containerd-logger-5vfmm
STEP: failed to find events of Pod "kube-scheduler-md-scale-yzpnko-control-plane-hcxhq"
STEP: Fetching activity logs took 2.497410754s
STEP: Dumping all the Cluster API resources in the "md-scale-tcemkb" namespace
STEP: Deleting cluster md-scale-tcemkb/md-scale-yzpnko
STEP: Deleting cluster md-scale-yzpnko
INFO: Waiting for the Cluster md-scale-tcemkb/md-scale-yzpnko to be deleted
STEP: Waiting for cluster md-scale-yzpnko to be deleted
... skipping 67 lines ...
STEP: Dumping logs from the "node-drain-kt0ztq" workload cluster
STEP: Dumping workload cluster node-drain-8sz6z3/node-drain-kt0ztq logs
Sep 30 21:03:46.281: INFO: Collecting logs for Linux node node-drain-kt0ztq-control-plane-5jb8l in cluster node-drain-kt0ztq in namespace node-drain-8sz6z3

Sep 30 21:10:20.004: INFO: Collecting boot logs for AzureMachine node-drain-kt0ztq-control-plane-5jb8l

Failed to get logs for machine node-drain-kt0ztq-control-plane-dkljx, cluster node-drain-8sz6z3/node-drain-kt0ztq: dialing public load balancer at node-drain-kt0ztq-fc22843f.eastus2.cloudapp.azure.com: dial tcp 20.22.127.162:22: connect: connection timed out
STEP: Dumping workload cluster node-drain-8sz6z3/node-drain-kt0ztq kube-system pod logs
STEP: Fetching kube-system pod logs took 408.111595ms
STEP: Dumping workload cluster node-drain-8sz6z3/node-drain-kt0ztq Azure activity log
STEP: Creating log watcher for controller kube-system/etcd-node-drain-kt0ztq-control-plane-5jb8l, container etcd
STEP: Collecting events for Pod kube-system/calico-node-ldzp2
STEP: Creating log watcher for controller kube-system/kube-controller-manager-node-drain-kt0ztq-control-plane-5jb8l, container kube-controller-manager
... skipping 93 lines ...

Sep 30 21:04:40.751: INFO: Collecting logs for Windows node quick-sta-fkmrv in cluster quick-start-d3sax9 in namespace quick-start-3kzoue

Sep 30 21:07:13.947: INFO: Attempting to copy file /c:/crashdumps.tar on node quick-sta-fkmrv to /logs/artifacts/clusters/quick-start-d3sax9/machines/quick-start-d3sax9-md-win-77b4dd89c4-74gwl/crashdumps.tar
Sep 30 21:07:15.214: INFO: Collecting boot logs for AzureMachine quick-start-d3sax9-md-win-fkmrv

Failed to get logs for machine quick-start-d3sax9-md-win-77b4dd89c4-74gwl, cluster quick-start-3kzoue/quick-start-d3sax9: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1]
Sep 30 21:07:16.278: INFO: Collecting logs for Windows node quick-sta-8jttg in cluster quick-start-d3sax9 in namespace quick-start-3kzoue

Sep 30 21:09:40.253: INFO: Attempting to copy file /c:/crashdumps.tar on node quick-sta-8jttg to /logs/artifacts/clusters/quick-start-d3sax9/machines/quick-start-d3sax9-md-win-77b4dd89c4-9jhsk/crashdumps.tar
Sep 30 21:09:41.615: INFO: Collecting boot logs for AzureMachine quick-start-d3sax9-md-win-8jttg

Failed to get logs for machine quick-start-d3sax9-md-win-77b4dd89c4-9jhsk, cluster quick-start-3kzoue/quick-start-d3sax9: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1]
STEP: Dumping workload cluster quick-start-3kzoue/quick-start-d3sax9 kube-system pod logs
STEP: Collecting events for Pod kube-system/calico-node-windows-lmj6p
STEP: Collecting events for Pod kube-system/calico-node-rkmtg
STEP: Collecting events for Pod kube-system/calico-node-windows-dxxgz
STEP: Collecting events for Pod kube-system/coredns-64897985d-9dg5j
STEP: Creating log watcher for controller kube-system/calico-node-windows-dxxgz, container calico-node-startup
... skipping 28 lines ...
STEP: Collecting events for Pod kube-system/csi-proxy-vk74p
STEP: Creating log watcher for controller kube-system/etcd-quick-start-d3sax9-control-plane-6mqq2, container etcd
STEP: Collecting events for Pod kube-system/etcd-quick-start-d3sax9-control-plane-6mqq2
STEP: Collecting events for Pod kube-system/kube-scheduler-quick-start-d3sax9-control-plane-6mqq2
STEP: Creating log watcher for controller kube-system/kube-proxy-6ghp5, container kube-proxy
STEP: Creating log watcher for controller kube-system/kube-controller-manager-quick-start-d3sax9-control-plane-6mqq2, container kube-controller-manager
STEP: failed to find events of Pod "kube-scheduler-quick-start-d3sax9-control-plane-6mqq2"
STEP: Collecting events for Pod kube-system/kube-controller-manager-quick-start-d3sax9-control-plane-6mqq2
STEP: Collecting events for Pod kube-system/kube-proxy-6ghp5
STEP: failed to find events of Pod "kube-controller-manager-quick-start-d3sax9-control-plane-6mqq2"
STEP: Collecting events for Pod kube-system/kube-apiserver-quick-start-d3sax9-control-plane-6mqq2
STEP: Fetching activity logs took 2.926207384s
STEP: Dumping all the Cluster API resources in the "quick-start-3kzoue" namespace
STEP: Deleting cluster quick-start-3kzoue/quick-start-d3sax9
STEP: Deleting cluster quick-start-d3sax9
INFO: Waiting for the Cluster quick-start-3kzoue/quick-start-d3sax9 to be deleted
... skipping 77 lines ...

STEP: Dumping workload cluster machine-pool-awir2i/machine-pool-ddwnge kube-system pod logs
STEP: Fetching kube-system pod logs took 423.019363ms
STEP: Collecting events for Pod kube-system/calico-node-cwh2d
STEP: Collecting events for Pod kube-system/kube-proxy-windows-kpgrf
STEP: Collecting events for Pod kube-system/kube-controller-manager-machine-pool-ddwnge-control-plane-nwqtn
STEP: failed to find events of Pod "kube-controller-manager-machine-pool-ddwnge-control-plane-nwqtn"
STEP: Collecting events for Pod kube-system/etcd-machine-pool-ddwnge-control-plane-nwqtn
STEP: Creating log watcher for controller kube-system/kube-scheduler-machine-pool-ddwnge-control-plane-nwqtn, container kube-scheduler
STEP: Collecting events for Pod kube-system/kube-scheduler-machine-pool-ddwnge-control-plane-nwqtn
STEP: failed to find events of Pod "etcd-machine-pool-ddwnge-control-plane-nwqtn"
STEP: Creating log watcher for controller kube-system/calico-kube-controllers-85f479877b-bvjmq, container calico-kube-controllers
STEP: failed to find events of Pod "kube-scheduler-machine-pool-ddwnge-control-plane-nwqtn"
STEP: Creating log watcher for controller kube-system/calico-node-windows-8wqrw, container calico-node-felix
STEP: Creating log watcher for controller kube-system/kube-proxy-2l5br, container kube-proxy
STEP: Creating log watcher for controller kube-system/calico-node-qklz9, container calico-node
STEP: Creating log watcher for controller kube-system/coredns-64897985d-n9jst, container coredns
STEP: Collecting events for Pod kube-system/coredns-64897985d-n9jst
STEP: Creating log watcher for controller kube-system/coredns-64897985d-nrdhv, container coredns
... skipping 4 lines ...
STEP: Creating log watcher for controller kube-system/etcd-machine-pool-ddwnge-control-plane-nwqtn, container etcd
STEP: Creating log watcher for controller kube-system/calico-node-cwh2d, container calico-node
STEP: Dumping workload cluster machine-pool-awir2i/machine-pool-ddwnge Azure activity log
STEP: Collecting events for Pod kube-system/coredns-64897985d-nrdhv
STEP: Creating log watcher for controller kube-system/kube-apiserver-machine-pool-ddwnge-control-plane-nwqtn, container kube-apiserver
STEP: Collecting events for Pod kube-system/kube-apiserver-machine-pool-ddwnge-control-plane-nwqtn
STEP: failed to find events of Pod "kube-apiserver-machine-pool-ddwnge-control-plane-nwqtn"
STEP: Collecting events for Pod kube-system/kube-proxy-2l5br
STEP: Creating log watcher for controller kube-system/kube-proxy-bxzgl, container kube-proxy
STEP: Creating log watcher for controller kube-system/kube-controller-manager-machine-pool-ddwnge-control-plane-nwqtn, container kube-controller-manager
STEP: Collecting events for Pod kube-system/kube-proxy-bxzgl
STEP: Creating log watcher for controller kube-system/kube-proxy-windows-kpgrf, container kube-proxy
STEP: Error starting logs stream for pod kube-system/kube-proxy-windows-kpgrf, container kube-proxy: pods "win-p-win000002" not found
STEP: Error starting logs stream for pod kube-system/calico-node-windows-8wqrw, container calico-node-startup: pods "win-p-win000002" not found
STEP: Error starting logs stream for pod kube-system/calico-node-qklz9, container calico-node: pods "machine-pool-ddwnge-mp-0000002" not found
STEP: Error starting logs stream for pod kube-system/calico-node-windows-8wqrw, container calico-node-felix: pods "win-p-win000002" not found
STEP: Error starting logs stream for pod kube-system/kube-proxy-bxzgl, container kube-proxy: pods "machine-pool-ddwnge-mp-0000002" not found
STEP: Fetching activity logs took 2.206005569s
STEP: Dumping all the Cluster API resources in the "machine-pool-awir2i" namespace
STEP: Deleting cluster machine-pool-awir2i/machine-pool-ddwnge
STEP: Deleting cluster machine-pool-ddwnge
INFO: Waiting for the Cluster machine-pool-awir2i/machine-pool-ddwnge to be deleted
STEP: Waiting for cluster machine-pool-ddwnge to be deleted
... skipping 9 lines ...
/home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/test/e2e/capi_test.go:45
  Should successfully exercise machine pools
  /home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/test/e2e/capi_test.go:171
    Should successfully create a cluster with machine pool machines
    /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.2.1/e2e/machine_pool.go:77
------------------------------
{"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:165","func":"k8s.io/test-infra/prow/entrypoint.Options.ExecuteProcess","level":"error","msg":"Process did not finish before 4h0m0s timeout","severity":"error","time":"2022-10-01T00:38:12Z"}
++ early_exit_handler
++ '[' -n 159 ']'
++ kill -TERM 159
++ cleanup_dind
++ [[ true == \t\r\u\e ]]
++ echo 'Cleaning up after docker'
... skipping 12 lines ...
Cleaning up after docker
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Stopping Docker: dockerProgram process in pidfile '/var/run/docker-ssd.pid', 1 process(es), refused to die.
================================================================================
Done cleaning up after docker in docker.
All sensitive variables are redacted
{"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:255","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Process did not exit before 15m0s grace period","severity":"error","time":"2022-10-01T00:53:12Z"}
{"component":"entrypoint","error":"os: process already finished","file":"k8s.io/test-infra/prow/entrypoint/run.go:257","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Could not kill process after grace period","severity":"error","time":"2022-10-01T00:53:12Z"}