This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 0 failed / 7 succeeded
Started2023-01-07 16:56
Elapsed4h15m
Revisionrelease-1.5

No Test Failures!


Show 7 Passed Tests

Show 4 Skipped Tests

Error lines from build-log.txt

... skipping 578 lines ...
STEP: Collecting events for Pod kube-system/coredns-bd6b6df9f-hjkjp
STEP: Collecting events for Pod kube-system/calico-kube-controllers-85f479877b-5wfq6
STEP: Creating log watcher for controller kube-system/calico-node-2b9d6, container calico-node
STEP: Creating log watcher for controller kube-system/kube-controller-manager-kcp-adoption-t1ln8v-control-plane-0, container kube-controller-manager
STEP: Collecting events for Pod kube-system/kube-proxy-5zbzq
STEP: Collecting events for Pod kube-system/kube-controller-manager-kcp-adoption-t1ln8v-control-plane-0
STEP: failed to find events of Pod "kube-controller-manager-kcp-adoption-t1ln8v-control-plane-0"
STEP: Creating log watcher for controller kube-system/kube-proxy-5zbzq, container kube-proxy
STEP: Creating log watcher for controller kube-system/kube-scheduler-kcp-adoption-t1ln8v-control-plane-0, container kube-scheduler
STEP: Creating log watcher for controller kube-system/coredns-bd6b6df9f-lf2tm, container coredns
STEP: Collecting events for Pod kube-system/coredns-bd6b6df9f-lf2tm
STEP: Collecting events for Pod kube-system/kube-scheduler-kcp-adoption-t1ln8v-control-plane-0
STEP: failed to find events of Pod "kube-scheduler-kcp-adoption-t1ln8v-control-plane-0"
STEP: Creating log watcher for controller kube-system/coredns-bd6b6df9f-hjkjp, container coredns
STEP: Collecting events for Pod kube-system/calico-node-2b9d6
STEP: Collecting events for Pod kube-system/etcd-kcp-adoption-t1ln8v-control-plane-0
STEP: Creating log watcher for controller kube-system/calico-kube-controllers-85f479877b-5wfq6, container calico-kube-controllers
STEP: Creating log watcher for controller kube-system/etcd-kcp-adoption-t1ln8v-control-plane-0, container etcd
STEP: Creating log watcher for controller kube-system/kube-apiserver-kcp-adoption-t1ln8v-control-plane-0, container kube-apiserver
STEP: Collecting events for Pod kube-system/kube-apiserver-kcp-adoption-t1ln8v-control-plane-0
STEP: failed to find events of Pod "kube-apiserver-kcp-adoption-t1ln8v-control-plane-0"
STEP: failed to find events of Pod "etcd-kcp-adoption-t1ln8v-control-plane-0"
STEP: Fetching activity logs took 1.693040462s
STEP: Dumping all the Cluster API resources in the "kcp-adoption-yrfqz2" namespace
STEP: Deleting cluster kcp-adoption-yrfqz2/kcp-adoption-t1ln8v
STEP: Deleting cluster kcp-adoption-t1ln8v
INFO: Waiting for the Cluster kcp-adoption-yrfqz2/kcp-adoption-t1ln8v to be deleted
STEP: Waiting for cluster kcp-adoption-t1ln8v to be deleted
... skipping 16 lines ...
  Should pivot the bootstrap cluster to a self-hosted cluster
  /home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/test/e2e/azure_selfhosted.go:107

INFO: "Should pivot the bootstrap cluster to a self-hosted cluster" started at Sat, 07 Jan 2023 17:04:45 UTC on Ginkgo node 8 of 10
STEP: Creating namespace "self-hosted" for hosting the cluster
Jan  7 17:04:45.248: INFO: starting to create namespace for hosting the "self-hosted" test spec
2023/01/07 17:04:45 failed trying to get namespace (self-hosted):namespaces "self-hosted" not found
INFO: Creating namespace self-hosted
INFO: Creating event watcher for namespace "self-hosted"
STEP: Creating a workload cluster
INFO: Creating the workload cluster with name "self-hosted-gzif6t" using the "management" template (Kubernetes v1.23.15, 1 control-plane machines, 1 worker machines)
INFO: Getting the cluster template yaml
INFO: clusterctl config cluster self-hosted-gzif6t --infrastructure (default) --kubernetes-version v1.23.15 --control-plane-machine-count 1 --worker-machine-count 1 --flavor management
... skipping 167 lines ...
STEP: Creating log watcher for controller kube-system/calico-kube-controllers-85f479877b-w6d9z, container calico-kube-controllers
STEP: Creating log watcher for controller kube-system/kube-scheduler-mhc-remediation-vsfumg-control-plane-87dk5, container kube-scheduler
STEP: Collecting events for Pod kube-system/kube-proxy-cf26d
STEP: Collecting events for Pod kube-system/coredns-bd6b6df9f-8rcx9
STEP: Collecting events for Pod kube-system/kube-scheduler-mhc-remediation-vsfumg-control-plane-87dk5
STEP: Creating log watcher for controller kube-system/kube-proxy-wftz5, container kube-proxy
STEP: failed to find events of Pod "kube-scheduler-mhc-remediation-vsfumg-control-plane-87dk5"
STEP: Dumping workload cluster mhc-remediation-ay6jge/mhc-remediation-vsfumg Azure activity log
STEP: Collecting events for Pod kube-system/kube-proxy-wftz5
STEP: Collecting events for Pod kube-system/etcd-mhc-remediation-vsfumg-control-plane-87dk5
STEP: Creating log watcher for controller kube-system/calico-node-99vxp, container calico-node
STEP: failed to find events of Pod "etcd-mhc-remediation-vsfumg-control-plane-87dk5"
STEP: Collecting events for Pod kube-system/calico-kube-controllers-85f479877b-w6d9z
STEP: Creating log watcher for controller kube-system/kube-apiserver-mhc-remediation-vsfumg-control-plane-87dk5, container kube-apiserver
STEP: Collecting events for Pod kube-system/calico-node-99vxp
STEP: Creating log watcher for controller kube-system/calico-node-qbpxg, container calico-node
STEP: Collecting events for Pod kube-system/calico-node-qbpxg
STEP: Collecting events for Pod kube-system/kube-apiserver-mhc-remediation-vsfumg-control-plane-87dk5
STEP: Creating log watcher for controller kube-system/kube-controller-manager-mhc-remediation-vsfumg-control-plane-87dk5, container kube-controller-manager
STEP: Collecting events for Pod kube-system/coredns-bd6b6df9f-vvtvk
STEP: Creating log watcher for controller kube-system/etcd-mhc-remediation-vsfumg-control-plane-87dk5, container etcd
STEP: Creating log watcher for controller kube-system/coredns-bd6b6df9f-vvtvk, container coredns
STEP: Creating log watcher for controller kube-system/coredns-bd6b6df9f-8rcx9, container coredns
STEP: Collecting events for Pod kube-system/kube-controller-manager-mhc-remediation-vsfumg-control-plane-87dk5
STEP: failed to find events of Pod "kube-controller-manager-mhc-remediation-vsfumg-control-plane-87dk5"
STEP: Creating log watcher for controller kube-system/kube-proxy-cf26d, container kube-proxy
STEP: Fetching activity logs took 1.840465516s
STEP: Dumping all the Cluster API resources in the "mhc-remediation-ay6jge" namespace
STEP: Deleting cluster mhc-remediation-ay6jge/mhc-remediation-vsfumg
STEP: Deleting cluster mhc-remediation-vsfumg
INFO: Waiting for the Cluster mhc-remediation-ay6jge/mhc-remediation-vsfumg to be deleted
... skipping 237 lines ...
STEP: Collecting events for Pod kube-system/kube-scheduler-node-drain-8bj900-control-plane-ldszs
STEP: Collecting events for Pod kube-system/etcd-node-drain-8bj900-control-plane-ldszs
STEP: Collecting events for Pod kube-system/kube-proxy-zp749
STEP: Collecting events for Pod kube-system/coredns-bd6b6df9f-qxwmg
STEP: Collecting events for Pod kube-system/calico-kube-controllers-85f479877b-lt4cs
STEP: Collecting events for Pod kube-system/kube-scheduler-node-drain-8bj900-control-plane-l7tz8
STEP: Error starting logs stream for pod kube-system/kube-apiserver-node-drain-8bj900-control-plane-l7tz8, container kube-apiserver: pods "node-drain-8bj900-control-plane-l7tz8" not found
STEP: Error starting logs stream for pod kube-system/kube-controller-manager-node-drain-8bj900-control-plane-l7tz8, container kube-controller-manager: pods "node-drain-8bj900-control-plane-l7tz8" not found
STEP: Error starting logs stream for pod kube-system/calico-node-m6gcz, container calico-node: pods "node-drain-8bj900-control-plane-l7tz8" not found
STEP: Error starting logs stream for pod kube-system/etcd-node-drain-8bj900-control-plane-l7tz8, container etcd: pods "node-drain-8bj900-control-plane-l7tz8" not found
STEP: Error starting logs stream for pod kube-system/kube-scheduler-node-drain-8bj900-control-plane-l7tz8, container kube-scheduler: pods "node-drain-8bj900-control-plane-l7tz8" not found
STEP: Error starting logs stream for pod kube-system/kube-proxy-zp749, container kube-proxy: pods "node-drain-8bj900-control-plane-l7tz8" not found
STEP: Fetching activity logs took 2.32501411s
STEP: Dumping all the Cluster API resources in the "node-drain-t9hyiy" namespace
STEP: Deleting cluster node-drain-t9hyiy/node-drain-8bj900
STEP: Deleting cluster node-drain-8bj900
INFO: Waiting for the Cluster node-drain-t9hyiy/node-drain-8bj900 to be deleted
STEP: Waiting for cluster node-drain-8bj900 to be deleted
... skipping 80 lines ...
STEP: Creating log watcher for controller kube-system/kube-proxy-s7zqz, container kube-proxy
STEP: Fetching kube-system pod logs took 408.72776ms
STEP: Dumping workload cluster machine-pool-n32b9k/machine-pool-3t7lje Azure activity log
STEP: Collecting events for Pod kube-system/kube-scheduler-machine-pool-3t7lje-control-plane-8r7cl
STEP: Collecting events for Pod kube-system/kube-controller-manager-machine-pool-3t7lje-control-plane-8r7cl
STEP: Creating log watcher for controller kube-system/calico-node-pblv2, container calico-node
STEP: failed to find events of Pod "kube-scheduler-machine-pool-3t7lje-control-plane-8r7cl"
STEP: Creating log watcher for controller kube-system/calico-kube-controllers-85f479877b-wcfxr, container calico-kube-controllers
STEP: failed to find events of Pod "kube-controller-manager-machine-pool-3t7lje-control-plane-8r7cl"
STEP: Collecting events for Pod kube-system/kube-apiserver-machine-pool-3t7lje-control-plane-8r7cl
STEP: Creating log watcher for controller kube-system/calico-node-windows-bm8rc, container calico-node-startup
STEP: Collecting events for Pod kube-system/calico-kube-controllers-85f479877b-wcfxr
STEP: Collecting events for Pod kube-system/kube-proxy-windows-pcg4l
STEP: Collecting events for Pod kube-system/kube-proxy-s7zqz
STEP: Creating log watcher for controller kube-system/kube-controller-manager-machine-pool-3t7lje-control-plane-8r7cl, container kube-controller-manager
... skipping 5 lines ...
STEP: Collecting events for Pod kube-system/coredns-bd6b6df9f-gvhxc
STEP: Creating log watcher for controller kube-system/etcd-machine-pool-3t7lje-control-plane-8r7cl, container etcd
STEP: Creating log watcher for controller kube-system/coredns-bd6b6df9f-9r25f, container coredns
STEP: Collecting events for Pod kube-system/kube-proxy-rv29n
STEP: Collecting events for Pod kube-system/coredns-bd6b6df9f-9r25f
STEP: Collecting events for Pod kube-system/etcd-machine-pool-3t7lje-control-plane-8r7cl
STEP: failed to find events of Pod "etcd-machine-pool-3t7lje-control-plane-8r7cl"
STEP: Collecting events for Pod kube-system/calico-node-pblv2
STEP: Creating log watcher for controller kube-system/calico-node-vjgh9, container calico-node
STEP: Creating log watcher for controller kube-system/kube-apiserver-machine-pool-3t7lje-control-plane-8r7cl, container kube-apiserver
STEP: Error starting logs stream for pod kube-system/calico-node-windows-bm8rc, container calico-node-felix: pods "win-p-win000002" not found
STEP: Error starting logs stream for pod kube-system/kube-proxy-s7zqz, container kube-proxy: pods "kube-proxy-s7zqz" not found
STEP: Error starting logs stream for pod kube-system/calico-node-pblv2, container calico-node: pods "calico-node-pblv2" not found
STEP: Error starting logs stream for pod kube-system/kube-proxy-windows-pcg4l, container kube-proxy: pods "win-p-win000002" not found
STEP: Error starting logs stream for pod kube-system/calico-node-windows-bm8rc, container calico-node-startup: pods "win-p-win000002" not found
STEP: Fetching activity logs took 1.782335675s
STEP: Dumping all the Cluster API resources in the "machine-pool-n32b9k" namespace
STEP: Deleting cluster machine-pool-n32b9k/machine-pool-3t7lje
STEP: Deleting cluster machine-pool-3t7lje
INFO: Waiting for the Cluster machine-pool-n32b9k/machine-pool-3t7lje to be deleted
STEP: Waiting for cluster machine-pool-3t7lje to be deleted
... skipping 72 lines ...

Jan  7 17:13:15.128: INFO: Collecting logs for Windows node quick-sta-hmg97 in cluster quick-start-ry5gq6 in namespace quick-start-m06ubp

Jan  7 17:15:56.902: INFO: Attempting to copy file /c:/crashdumps.tar on node quick-sta-hmg97 to /logs/artifacts/clusters/quick-start-ry5gq6/machines/quick-start-ry5gq6-md-win-8457b57f6c-5wznx/crashdumps.tar
Jan  7 17:15:58.704: INFO: Collecting boot logs for AzureMachine quick-start-ry5gq6-md-win-hmg97

Failed to get logs for machine quick-start-ry5gq6-md-win-8457b57f6c-5wznx, cluster quick-start-m06ubp/quick-start-ry5gq6: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1]
Jan  7 17:15:59.653: INFO: Collecting logs for Windows node quick-sta-dxhbc in cluster quick-start-ry5gq6 in namespace quick-start-m06ubp

Jan  7 17:18:35.785: INFO: Attempting to copy file /c:/crashdumps.tar on node quick-sta-dxhbc to /logs/artifacts/clusters/quick-start-ry5gq6/machines/quick-start-ry5gq6-md-win-8457b57f6c-xjvzb/crashdumps.tar
Jan  7 17:18:37.573: INFO: Collecting boot logs for AzureMachine quick-start-ry5gq6-md-win-dxhbc

Failed to get logs for machine quick-start-ry5gq6-md-win-8457b57f6c-xjvzb, cluster quick-start-m06ubp/quick-start-ry5gq6: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1]
STEP: Dumping workload cluster quick-start-m06ubp/quick-start-ry5gq6 kube-system pod logs
STEP: Collecting events for Pod kube-system/calico-kube-controllers-85f479877b-c2nhv
STEP: Creating log watcher for controller kube-system/calico-node-5xrtq, container calico-node
STEP: Creating log watcher for controller kube-system/calico-node-windows-9rb2s, container calico-node-startup
STEP: Collecting events for Pod kube-system/calico-node-windows-2fgxt
STEP: Creating log watcher for controller kube-system/etcd-quick-start-ry5gq6-control-plane-9fplt, container etcd
STEP: Collecting events for Pod kube-system/calico-node-6q2x9
STEP: Creating log watcher for controller kube-system/calico-node-windows-2fgxt, container calico-node-startup
STEP: Collecting events for Pod kube-system/calico-node-5xrtq
STEP: Creating log watcher for controller kube-system/calico-node-6q2x9, container calico-node
STEP: Collecting events for Pod kube-system/etcd-quick-start-ry5gq6-control-plane-9fplt
STEP: failed to find events of Pod "etcd-quick-start-ry5gq6-control-plane-9fplt"
STEP: Creating log watcher for controller kube-system/kube-apiserver-quick-start-ry5gq6-control-plane-9fplt, container kube-apiserver
STEP: Collecting events for Pod kube-system/kube-scheduler-quick-start-ry5gq6-control-plane-9fplt
STEP: Creating log watcher for controller kube-system/calico-kube-controllers-85f479877b-c2nhv, container calico-kube-controllers
STEP: failed to find events of Pod "kube-scheduler-quick-start-ry5gq6-control-plane-9fplt"
STEP: Creating log watcher for controller kube-system/kube-controller-manager-quick-start-ry5gq6-control-plane-9fplt, container kube-controller-manager
STEP: Collecting events for Pod kube-system/kube-proxy-windows-bf2ld
STEP: Creating log watcher for controller kube-system/calico-node-windows-2fgxt, container calico-node-felix
STEP: Collecting events for Pod kube-system/kube-controller-manager-quick-start-ry5gq6-control-plane-9fplt
STEP: failed to find events of Pod "kube-controller-manager-quick-start-ry5gq6-control-plane-9fplt"
STEP: Creating log watcher for controller kube-system/kube-proxy-kp44n, container kube-proxy
STEP: Creating log watcher for controller kube-system/kube-proxy-wn6mv, container kube-proxy
STEP: Creating log watcher for controller kube-system/calico-node-windows-9rb2s, container calico-node-felix
STEP: Collecting events for Pod kube-system/kube-apiserver-quick-start-ry5gq6-control-plane-9fplt
STEP: Creating log watcher for controller kube-system/kube-proxy-windows-mpk7h, container kube-proxy
STEP: Collecting events for Pod kube-system/kube-proxy-wn6mv
... skipping 13 lines ...
STEP: Collecting events for Pod kube-system/coredns-bd6b6df9f-qp5nt
STEP: Creating log watcher for controller kube-system/csi-proxy-qhxqg, container csi-proxy
STEP: Creating log watcher for controller kube-system/csi-proxy-hd2vl, container csi-proxy
STEP: Collecting events for Pod kube-system/containerd-logger-pl6mm
STEP: Creating log watcher for controller kube-system/coredns-bd6b6df9f-d2ckc, container coredns
STEP: Collecting events for Pod kube-system/csi-proxy-qhxqg
STEP: failed to find events of Pod "kube-apiserver-quick-start-ry5gq6-control-plane-9fplt"
STEP: Fetching activity logs took 1.963178134s
STEP: Dumping all the Cluster API resources in the "quick-start-m06ubp" namespace
STEP: Deleting cluster quick-start-m06ubp/quick-start-ry5gq6
STEP: Deleting cluster quick-start-ry5gq6
INFO: Waiting for the Cluster quick-start-m06ubp/quick-start-ry5gq6 to be deleted
STEP: Waiting for cluster quick-start-ry5gq6 to be deleted
... skipping 78 lines ...

Jan  7 17:15:38.480: INFO: Collecting logs for Windows node md-scale-f55zw in cluster md-scale-kagl7r in namespace md-scale-uqg72n

Jan  7 17:18:16.489: INFO: Attempting to copy file /c:/crashdumps.tar on node md-scale-f55zw to /logs/artifacts/clusters/md-scale-kagl7r/machines/md-scale-kagl7r-md-win-95c785855-52ccm/crashdumps.tar
Jan  7 17:18:18.022: INFO: Collecting boot logs for AzureMachine md-scale-kagl7r-md-win-f55zw

Failed to get logs for machine md-scale-kagl7r-md-win-95c785855-52ccm, cluster md-scale-uqg72n/md-scale-kagl7r: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1]
Jan  7 17:18:18.943: INFO: Collecting logs for Windows node md-scale-z8phb in cluster md-scale-kagl7r in namespace md-scale-uqg72n

Jan  7 17:20:48.738: INFO: Attempting to copy file /c:/crashdumps.tar on node md-scale-z8phb to /logs/artifacts/clusters/md-scale-kagl7r/machines/md-scale-kagl7r-md-win-95c785855-zctt9/crashdumps.tar
Jan  7 17:20:50.561: INFO: Collecting boot logs for AzureMachine md-scale-kagl7r-md-win-z8phb

Failed to get logs for machine md-scale-kagl7r-md-win-95c785855-zctt9, cluster md-scale-uqg72n/md-scale-kagl7r: [running command "Get-Content "C:\\cni.log"": Process exited with status 1, running command "$p = 'c:\localdumps' ; if (Test-Path $p) { tar.exe -cvzf c:\crashdumps.tar $p *>&1 | %{ Write-Output "$_"} } else { Write-Host "No crash dumps found at $p" }": Process exited with status 1]
STEP: Dumping workload cluster md-scale-uqg72n/md-scale-kagl7r kube-system pod logs
STEP: Creating log watcher for controller kube-system/calico-kube-controllers-85f479877b-b7pgz, container calico-kube-controllers
STEP: Collecting events for Pod kube-system/calico-kube-controllers-85f479877b-b7pgz
STEP: Collecting events for Pod kube-system/containerd-logger-7hrwg
STEP: Collecting events for Pod kube-system/calico-node-kd84f
STEP: Collecting events for Pod kube-system/kube-scheduler-md-scale-kagl7r-control-plane-sz9k2
... skipping 52 lines ...
/home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/test/e2e/capi_test.go:45
  Should successfully scale out and scale in a MachineDeployment
  /home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/test/e2e/capi_test.go:183
    Should successfully scale a MachineDeployment up and down upon changes to the MachineDeployment replica count
    /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.2.1/e2e/md_scale.go:71
------------------------------
{"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:164","func":"k8s.io/test-infra/prow/entrypoint.Options.ExecuteProcess","level":"error","msg":"Process did not finish before 4h0m0s timeout","severity":"error","time":"2023-01-07T20:56:28Z"}
++ early_exit_handler
++ '[' -n 160 ']'
++ kill -TERM 160
++ cleanup_dind
++ [[ true == \t\r\u\e ]]
++ echo 'Cleaning up after docker'
... skipping 12 lines ...
Cleaning up after docker
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Stopping Docker: dockerProgram process in pidfile '/var/run/docker-ssd.pid', 1 process(es), refused to die.
================================================================================
Done cleaning up after docker in docker.
All sensitive variables are redacted
{"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:254","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Process did not exit before 15m0s grace period","severity":"error","time":"2023-01-07T21:11:28Z"}
{"component":"entrypoint","error":"os: process already finished","file":"k8s.io/test-infra/prow/entrypoint/run.go:256","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Could not kill process after grace period","severity":"error","time":"2023-01-07T21:11:28Z"}