This job view page is being replaced by Spyglass soon. Check out the new job view.
PRMirzaSikander: Not send availability zones as part of create for edge zones.
ResultFAILURE
Tests 1 failed / 35 succeeded
Started2021-07-16 07:57
Elapsed1h16m
Revision12f0aab9e350f1c8d62d85753eb8f3ac0dcb694d
Refs 708
job-versionv1.21.4-rc.0.1+1c29b7706e7642
kubetest-version
revisionv1.21.4-rc.0.1+1c29b7706e7642

Test Failures


kubetest DumpClusterLogs 1m26s

error running log collection script /root/tmp653352196/log-dump.sh: error during bash -c /root/tmp653352196/log-dump.sh: exit status 1
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 35 Passed Tests

Show 12 Skipped Tests

Error lines from build-log.txt

... skipping 226 lines ...
STEP: Getting agent nodes list
STEP: Get vmss names from node providerIDs
Jul 16 08:07:40.325: INFO: Got vmss names [k8s-agentpool1-16771286-vmss k8s-agentpool2-16771286-vmss]
STEP: Creating service annotation-test-k8s-agentpool1-16771286-vmss in namespace e2e-tests-service-vtxqk
Jul 16 08:07:40.535: INFO: Successfully created LoadBalancer service annotation-test-k8s-agentpool1-16771286-vmss in namespace e2e-tests-service-vtxqk
STEP: Waiting for service exposure
Jul 16 08:07:40.635: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:07:50.737: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:08:00.737: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:08:10.737: INFO: Exposure successfully, get external ip: 51.104.230.15
STEP: Getting public IPs in the resourceGroup kubetest-qhgwh1no
STEP: Getting public IP frontend configuration ID
STEP: Getting loadBalancer name from pipFrontendConfigurationID
Jul 16 08:08:11.256: INFO: Got loadBalancerName "kubetest-qhgwh1no"
STEP: Getting loadBalancer
STEP: Getting loadBalancer backendPoolID
STEP: Validating loadBalancer backendPool
STEP: Creating service annotation-test-k8s-agentpool2-16771286-vmss in namespace e2e-tests-service-vtxqk
Jul 16 08:08:11.564: INFO: Successfully created LoadBalancer service annotation-test-k8s-agentpool2-16771286-vmss in namespace e2e-tests-service-vtxqk
STEP: Waiting for service exposure
Jul 16 08:08:11.664: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:08:21.766: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:08:31.769: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:08:41.767: INFO: Exposure successfully, get external ip: 51.11.50.211
STEP: Getting public IPs in the resourceGroup kubetest-qhgwh1no
STEP: Getting public IP frontend configuration ID
STEP: Getting loadBalancer name from pipFrontendConfigurationID
Jul 16 08:08:41.926: INFO: Got loadBalancerName "k8s-agentpool2-16771286-vmss"
STEP: Getting loadBalancer
... skipping 17 lines ...
Jul 16 08:10:34.605: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:10:35.002: INFO: Creating deployment nsg-test
STEP: Creating a service and expose it
Jul 16 08:10:35.108: INFO: Creating service nsg-test in namespace e2e-tests-nsg-qjjpv
Jul 16 08:10:35.253: INFO: Successfully created LoadBalancer service nsg-test in namespace e2e-tests-nsg-qjjpv
Jul 16 08:10:35.253: INFO: Waiting service to expose...
Jul 16 08:10:35.353: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:10:45.455: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:10:55.455: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:11:05.455: INFO: Exposure successfully, get external ip: 20.90.82.185
STEP: Validating ip exists in Security Group
Jul 16 08:11:05.455: INFO: Getting virtual network list
Jul 16 08:11:06.011: INFO: got sg list, length = 1
Jul 16 08:11:06.011: INFO: Found target security rule
STEP: Validating network security group working
... skipping 25 lines ...
Jul 16 08:13:05.540: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:13:05.935: INFO: Creating deployment nsg-test
STEP: Creating service and wait it to expose
Jul 16 08:13:06.040: INFO: Creating service nsg-test in namespace e2e-tests-nsg-w68dv
Jul 16 08:13:06.164: INFO: Successfully created LoadBalancer service nsg-test in namespace e2e-tests-nsg-w68dv
Jul 16 08:13:06.164: INFO: Waiting service to expose...
Jul 16 08:13:06.268: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:13:16.371: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:13:26.372: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:13:36.370: INFO: Exposure successfully, get external ip: 51.143.173.220
STEP: Validating if the corresponding IP prefix existing in nsg
Jul 16 08:13:36.370: INFO: Getting virtual network list
Jul 16 08:13:36.737: INFO: got sg list, length = 1
Jul 16 08:13:36.844: INFO: Deleting namespace e2e-tests-nsg-w68dv

... skipping 11 lines ...
Jul 16 08:14:31.174: INFO: Creating a test namespace
Jul 16 08:14:31.283: INFO: Creating a kubernetes client
Jul 16 08:14:31.283: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:14:31.680: INFO: Creating deployment nsg-test
STEP: Creating a test service with the deny rule annotation but without `service.Spec.LoadBalancerSourceRanges`
STEP: Waiting for the service to expose
Jul 16 08:14:32.118: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:14:42.219: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:14:52.222: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:15:02.220: INFO: Exposure successfully, get external ip: 10.240.0.8
STEP: Checking if there is a deny_all rule
Jul 16 08:15:02.220: INFO: Getting virtual network list
Jul 16 08:15:02.480: INFO: got sg list, length = 1
STEP: Deleting the service
Jul 16 08:15:02.614: INFO: Deleting service nsg-test in namespace e2e-tests-nsg-f67sq
STEP: Creating a test service with the deny rule annotation and `service.Spec.LoadBalancerSourceRanges`
STEP: Waiting for the service to expose
Jul 16 08:15:35.037: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:15:45.139: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:15:55.139: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:16:05.139: INFO: Exposure successfully, get external ip: 10.240.0.8
STEP: Checking if there is a LoadBalancerSourceRanges rule
Jul 16 08:16:05.139: INFO: Getting virtual network list
Jul 16 08:16:05.328: INFO: got sg list, length = 1
STEP: Checking if there is a deny_all rule
Jul 16 08:16:05.434: INFO: Deleting namespace e2e-tests-nsg-f67sq
... skipping 14 lines ...
Jul 16 08:16:51.849: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:16:52.248: INFO: Creating deployment servicelb-test
Jul 16 08:16:52.524: INFO: Successfully created LoadBalancer service servicelb-test in namespace e2e-tests-service-lb-7gjcd
Jul 16 08:16:52.524: INFO: Creating public IP resource named service-lb-public-none-IPb4a1
Jul 16 08:16:57.079: INFO: PIP to 51.145.49.1
STEP: Waiting for exposure of the original service without assigned lb IP
Jul 16 08:16:57.179: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:17:07.283: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:17:17.284: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:17:27.281: INFO: Exposure successfully, get external ip: 51.145.51.229
STEP: Updating service to bound to specific public IP
Jul 16 08:17:27.281: INFO: will update IP to 51.145.49.1
Jul 16 08:17:27.604: INFO: still unmatched external IP, retry it in 10s seconds
Jul 16 08:17:37.708: INFO: still unmatched external IP, retry it in 10s seconds
Jul 16 08:17:47.705: INFO: still unmatched external IP, retry it in 10s seconds
Jul 16 08:17:57.707: INFO: still unmatched external IP, retry it in 10s seconds
Jul 16 08:18:07.707: INFO: still unmatched external IP, retry it in 10s seconds
Jul 16 08:18:17.708: INFO: still unmatched external IP, retry it in 10s seconds
Jul 16 08:18:27.707: INFO: Exposure successfully
STEP: Cleaning up
Jul 16 08:18:27.841: INFO: Deleting public IP resource named service-lb-public-none-IPb4a1 in resource group kubetest-qhgwh1no
Jul 16 08:18:28.410: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-none-IPb4a1 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a8206f91c4fa5475dbc401dffc0c80b1. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:18:30.844: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-none-IPb4a1 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a8206f91c4fa5475dbc401dffc0c80b1. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:18:32.808: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-none-IPb4a1 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a8206f91c4fa5475dbc401dffc0c80b1. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:18:34.824: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-none-IPb4a1 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a8206f91c4fa5475dbc401dffc0c80b1. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:18:36.808: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-none-IPb4a1 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a8206f91c4fa5475dbc401dffc0c80b1. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:18:38.802: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-none-IPb4a1 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a8206f91c4fa5475dbc401dffc0c80b1. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:18:40.924: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-none-IPb4a1 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a8206f91c4fa5475dbc401dffc0c80b1. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:18:42.948: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-none-IPb4a1 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a8206f91c4fa5475dbc401dffc0c80b1. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:18:44.796: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-none-IPb4a1 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a8206f91c4fa5475dbc401dffc0c80b1. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:18:46.796: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-none-IPb4a1 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a8206f91c4fa5475dbc401dffc0c80b1. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:18:48.959: INFO: Deleting namespace e2e-tests-service-lb-7gjcd

• [SLOW TEST:129.558 seconds]
Ensure LoadBalancer
/home/prow/go/src/sigs.k8s.io/cloud-provider-azure/tests/e2e/network/ensureloadbalancer.go:46
  should support assigning to specific IP when updating public service
... skipping 8 lines ...
Jul 16 08:19:01.407: INFO: Creating a kubernetes client
Jul 16 08:19:01.407: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:19:01.806: INFO: Creating deployment servicelb-test
Jul 16 08:19:01.913: INFO: Getting virtual network list
Jul 16 08:19:04.308: INFO: Successfully created LoadBalancer service servicelb-test in namespace e2e-tests-service-lb-vfthq
STEP: Waiting for exposure of internal service with specific IP
Jul 16 08:19:04.417: INFO: Fail to get ingress, retry it in 10s seconds
Jul 16 08:19:14.519: INFO: Fail to get ingress, retry it in 10s seconds
Jul 16 08:19:24.519: INFO: Fail to get ingress, retry it in 10s seconds
Jul 16 08:19:34.522: INFO: Exposure successfully
Jul 16 08:19:34.722: INFO: Events list:
Jul 16 08:19:34.722: INFO: 0. {{ } {servicelb-test-76bf687774-8fkq6.1692379820185daf  e2e-tests-service-lb-vfthq  1df33b84-18f1-4f55-9823-aa25dfccfd86 5894 0 2021-07-16 08:19:01 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-scheduler Update events.k8s.io/v1 2021-07-16 08:19:01 +0000 UTC FieldsV1 {"f:action":{},"f:eventTime":{},"f:note":{},"f:reason":{},"f:regarding":{"f:apiVersion":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:reportingController":{},"f:reportingInstance":{},"f:type":{}} }]} {Pod e2e-tests-service-lb-vfthq servicelb-test-76bf687774-8fkq6 ca51b0b4-8ff3-47ba-9185-0c6207977fe7 v1 5883 } Scheduled Successfully assigned e2e-tests-service-lb-vfthq/servicelb-test-76bf687774-8fkq6 to k8s-agentpool2-16771286-vmss000001 { } 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC 0 Normal 2021-07-16 08:19:01.94719 +0000 UTC nil Binding nil default-scheduler default-scheduler-k8s-master-16771286-0}
Jul 16 08:19:34.722: INFO: 1. {{ } {servicelb-test-76bf687774-8fkq6.1692379852823837  e2e-tests-service-lb-vfthq  fabf7b95-6fbc-45ba-a0da-d4a6f29ac32f 5907 0 2021-07-16 08:19:02 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2021-07-16 08:19:02 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{"f:apiVersion":{},"f:fieldPath":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]} {Pod e2e-tests-service-lb-vfthq servicelb-test-76bf687774-8fkq6 ca51b0b4-8ff3-47ba-9185-0c6207977fe7 v1 5891 spec.containers{test-app}} Pulling Pulling image "nginx:1.15" {kubelet k8s-agentpool2-16771286-vmss000001} 2021-07-16 08:19:02 +0000 UTC 2021-07-16 08:19:02 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }
Jul 16 08:19:34.722: INFO: 2. {{ } {servicelb-test-76bf687774-8fkq6.16923798900d4d8b  e2e-tests-service-lb-vfthq  6f8f3dc3-7817-4e0f-b890-1b7c47d3d5c4 5917 0 2021-07-16 08:19:03 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2021-07-16 08:19:03 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{"f:apiVersion":{},"f:fieldPath":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]} {Pod e2e-tests-service-lb-vfthq servicelb-test-76bf687774-8fkq6 ca51b0b4-8ff3-47ba-9185-0c6207977fe7 v1 5891 spec.containers{test-app}} Pulled Successfully pulled image "nginx:1.15" in 1.03251344s {kubelet k8s-agentpool2-16771286-vmss000001} 2021-07-16 08:19:03 +0000 UTC 2021-07-16 08:19:03 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }
Jul 16 08:19:34.722: INFO: 3. {{ } {servicelb-test-76bf687774-8fkq6.169237989561e2f4  e2e-tests-service-lb-vfthq  5f6f82e4-e06f-4229-934a-b8ae3d83b171 5923 0 2021-07-16 08:19:03 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2021-07-16 08:19:03 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{"f:apiVersion":{},"f:fieldPath":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]} {Pod e2e-tests-service-lb-vfthq servicelb-test-76bf687774-8fkq6 ca51b0b4-8ff3-47ba-9185-0c6207977fe7 v1 5891 spec.containers{test-app}} Created Created container test-app {kubelet k8s-agentpool2-16771286-vmss000001} 2021-07-16 08:19:03 +0000 UTC 2021-07-16 08:19:03 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }
... skipping 55 lines ...
Jul 16 08:21:25.355: INFO: Creating a kubernetes client
Jul 16 08:21:25.355: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:21:25.752: INFO: Creating deployment servicelb-test
Jul 16 08:21:25.972: INFO: Successfully created LoadBalancer service servicelb-test in namespace e2e-tests-service-lb-rcz6g
Jul 16 08:21:25.972: INFO: Creating public IP resource named service-lb-internal-none-public-IPdd2a
STEP: Waiting for exposure of the original service without assigned lb private IP
Jul 16 08:21:30.795: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:21:40.896: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:21:50.897: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:22:00.898: INFO: Exposure successfully, get external ip: 10.240.0.8
Jul 16 08:22:01.097: INFO: Events list:
Jul 16 08:22:01.097: INFO: 0. {{ } {servicelb-test-76bf687774-6hfnw.169237b9a17f8e69  e2e-tests-service-lb-rcz6g  daf1f5cf-f0f2-43d0-9539-fecb5da47f1b 6664 0 2021-07-16 08:21:25 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-scheduler Update events.k8s.io/v1 2021-07-16 08:21:25 +0000 UTC FieldsV1 {"f:action":{},"f:eventTime":{},"f:note":{},"f:reason":{},"f:regarding":{"f:apiVersion":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:reportingController":{},"f:reportingInstance":{},"f:type":{}} }]} {Pod e2e-tests-service-lb-rcz6g servicelb-test-76bf687774-6hfnw 646ee8b4-89f0-4def-b4b8-1a566fdf463b v1 6654 } Scheduled Successfully assigned e2e-tests-service-lb-rcz6g/servicelb-test-76bf687774-6hfnw to k8s-agentpool2-16771286-vmss000001 { } 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC 0 Normal 2021-07-16 08:21:25.852135 +0000 UTC nil Binding nil default-scheduler default-scheduler-k8s-master-16771286-0}
Jul 16 08:22:01.097: INFO: 1. {{ } {servicelb-test-76bf687774-6hfnw.169237b9d3d77df2  e2e-tests-service-lb-rcz6g  9365014e-201f-4ea0-a3eb-763275df5d60 6696 0 2021-07-16 08:21:26 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2021-07-16 08:21:26 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{"f:apiVersion":{},"f:fieldPath":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]} {Pod e2e-tests-service-lb-rcz6g servicelb-test-76bf687774-6hfnw 646ee8b4-89f0-4def-b4b8-1a566fdf463b v1 6657 spec.containers{test-app}} Pulling Pulling image "nginx:1.15" {kubelet k8s-agentpool2-16771286-vmss000001} 2021-07-16 08:21:26 +0000 UTC 2021-07-16 08:21:26 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }
Jul 16 08:22:01.097: INFO: 2. {{ } {servicelb-test-76bf687774-6hfnw.169237ba105870f6  e2e-tests-service-lb-rcz6g  561e6530-8802-4c4d-97e8-6d4dde455b39 6704 0 2021-07-16 08:21:27 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2021-07-16 08:21:27 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{"f:apiVersion":{},"f:fieldPath":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]} {Pod e2e-tests-service-lb-rcz6g servicelb-test-76bf687774-6hfnw 646ee8b4-89f0-4def-b4b8-1a566fdf463b v1 6657 spec.containers{test-app}} Pulled Successfully pulled image "nginx:1.15" in 1.015070779s {kubelet k8s-agentpool2-16771286-vmss000001} 2021-07-16 08:21:27 +0000 UTC 2021-07-16 08:21:27 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }
Jul 16 08:22:01.097: INFO: 3. {{ } {servicelb-test-76bf687774-6hfnw.169237ba15e7f397  e2e-tests-service-lb-rcz6g  32bfc864-6f86-43f5-8359-819f53110369 6708 0 2021-07-16 08:21:27 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2021-07-16 08:21:27 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{"f:apiVersion":{},"f:fieldPath":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]} {Pod e2e-tests-service-lb-rcz6g servicelb-test-76bf687774-6hfnw 646ee8b4-89f0-4def-b4b8-1a566fdf463b v1 6657 spec.containers{test-app}} Created Created container test-app {kubelet k8s-agentpool2-16771286-vmss000001} 2021-07-16 08:21:27 +0000 UTC 2021-07-16 08:21:27 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }
... skipping 34 lines ...
Jul 16 08:22:31.507: INFO: still unmatched external IP, retry it in 10s seconds
Jul 16 08:22:41.506: INFO: still unmatched external IP, retry it in 10s seconds
Jul 16 08:22:51.514: INFO: still unmatched external IP, retry it in 10s seconds
Jul 16 08:23:01.511: INFO: Exposure successfully
STEP: Cleaning up
Jul 16 08:23:01.637: INFO: Deleting public IP resource named service-lb-internal-none-public-IPdd2a in resource group kubetest-qhgwh1no
Jul 16 08:23:02.933: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-internal-none-public-IPdd2a can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a2c4643a57a2b4886a5c65f1494b1b22. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:23:05.315: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-internal-none-public-IPdd2a can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a2c4643a57a2b4886a5c65f1494b1b22. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:23:07.320: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-internal-none-public-IPdd2a can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a2c4643a57a2b4886a5c65f1494b1b22. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:23:09.310: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-internal-none-public-IPdd2a can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a2c4643a57a2b4886a5c65f1494b1b22. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:23:11.314: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-internal-none-public-IPdd2a can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a2c4643a57a2b4886a5c65f1494b1b22. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:23:13.315: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-internal-none-public-IPdd2a can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a2c4643a57a2b4886a5c65f1494b1b22. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:23:15.320: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-internal-none-public-IPdd2a can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a2c4643a57a2b4886a5c65f1494b1b22. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:23:17.319: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-internal-none-public-IPdd2a can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a2c4643a57a2b4886a5c65f1494b1b22. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:23:19.324: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-internal-none-public-IPdd2a can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a2c4643a57a2b4886a5c65f1494b1b22. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:23:21.313: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-internal-none-public-IPdd2a can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a2c4643a57a2b4886a5c65f1494b1b22. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:23:23.313: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-internal-none-public-IPdd2a can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a2c4643a57a2b4886a5c65f1494b1b22. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:23:25.365: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-internal-none-public-IPdd2a can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a2c4643a57a2b4886a5c65f1494b1b22. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:23:27.497: INFO: Deleting namespace e2e-tests-service-lb-rcz6g

• [SLOW TEST:138.562 seconds]
Ensure LoadBalancer
/home/prow/go/src/sigs.k8s.io/cloud-provider-azure/tests/e2e/network/ensureloadbalancer.go:46
  should support updating an internal service to a public service with assigned IP
... skipping 8 lines ...
Jul 16 08:23:43.931: INFO: Creating a kubernetes client
Jul 16 08:23:43.931: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:23:44.326: INFO: Creating deployment servicelb-test
Jul 16 08:23:44.431: INFO: Creating public IP resource named service-lb-public-remain5435
Jul 16 08:23:49.197: INFO: Successfully created LoadBalancer service servicelb-test in namespace e2e-tests-service-lb-wr4m7
STEP: Waiting for exposure of the original service with assigned lb private IP
Jul 16 08:23:49.298: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:23:59.400: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:24:09.400: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:24:19.400: INFO: Exposure successfully, get external ip: 51.145.53.79
STEP: Update without changing the service and wait for a while
Jul 16 08:24:19.400: INFO: External IP is now 51.145.53.79
Jul 16 08:24:19.500: INFO: testlabel5435
Jul 16 08:24:19.711: INFO: External IP is still 51.145.53.79
Jul 16 08:24:29.813: INFO: External IP is still 51.145.53.79
... skipping 56 lines ...
Jul 16 08:33:59.813: INFO: External IP is still 51.145.53.79
Jul 16 08:34:09.814: INFO: External IP is still 51.145.53.79
Jul 16 08:34:19.813: INFO: External IP is still 51.145.53.79
Jul 16 08:34:19.913: INFO: External IP is still 51.145.53.79
STEP: Cleaning up
Jul 16 08:34:20.045: INFO: Deleting public IP resource named service-lb-public-remain5435 in resource group kubetest-qhgwh1no
Jul 16 08:34:20.974: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain5435 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/afc5edaf7e29f4d6cb1d2fe9bb83fd5c. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:34:23.371: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain5435 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/afc5edaf7e29f4d6cb1d2fe9bb83fd5c. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:34:25.374: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain5435 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/afc5edaf7e29f4d6cb1d2fe9bb83fd5c. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:34:27.420: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain5435 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/afc5edaf7e29f4d6cb1d2fe9bb83fd5c. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:34:29.375: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain5435 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/afc5edaf7e29f4d6cb1d2fe9bb83fd5c. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:34:31.439: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain5435 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/afc5edaf7e29f4d6cb1d2fe9bb83fd5c. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:34:33.371: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain5435 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/afc5edaf7e29f4d6cb1d2fe9bb83fd5c. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:34:35.373: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain5435 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/afc5edaf7e29f4d6cb1d2fe9bb83fd5c. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:34:37.378: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain5435 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/afc5edaf7e29f4d6cb1d2fe9bb83fd5c. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:34:39.390: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain5435 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/afc5edaf7e29f4d6cb1d2fe9bb83fd5c. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:34:41.382: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain5435 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/afc5edaf7e29f4d6cb1d2fe9bb83fd5c. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:34:43.373: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain5435 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/afc5edaf7e29f4d6cb1d2fe9bb83fd5c. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:34:45.376: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain5435 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/afc5edaf7e29f4d6cb1d2fe9bb83fd5c. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:34:47.539: INFO: Deleting namespace e2e-tests-service-lb-wr4m7

• [SLOW TEST:680.034 seconds]
Ensure LoadBalancer
/home/prow/go/src/sigs.k8s.io/cloud-provider-azure/tests/e2e/network/ensureloadbalancer.go:46
  should have no operation since no change in service when update [Slow]
... skipping 6 lines ...
Jul 16 08:35:03.847: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:35:03.848: INFO: Creating a test namespace
Jul 16 08:35:03.954: INFO: Creating a kubernetes client
Jul 16 08:35:03.954: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:35:04.350: INFO: Creating deployment servicelb-test
Jul 16 08:35:04.454: INFO: Creating public IP resource named service-lb-public-remain2e57
Jul 16 08:35:09.375: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:35:19.477: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:35:29.477: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:35:39.477: INFO: Exposure successfully, get external ip: 51.11.48.73
Jul 16 08:35:39.477: INFO: Successfully created LoadBalancer service1 in namespace e2e-tests-service-lb-xhhsn with IP 51.11.48.73
Jul 16 08:35:39.702: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:35:49.804: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:35:59.805: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:36:09.804: INFO: Exposure successfully, get external ip: 51.11.48.73
Jul 16 08:36:09.804: INFO: Successfully created LoadBalancer service2 in namespace e2e-tests-service-lb-xhhsn with IP 51.11.48.73
STEP: Cleaning up
Jul 16 08:36:10.065: INFO: Deleting public IP resource named service-lb-public-remain2e57 in resource group kubetest-qhgwh1no
Jul 16 08:36:10.815: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:13.212: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:15.239: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:17.209: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:19.245: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:21.219: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:23.230: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:25.212: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:27.289: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:29.217: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:31.202: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:33.229: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:35.205: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:37.206: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:39.204: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:41.227: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:43.216: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:45.219: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:47.241: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:49.279: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:51.219: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:53.206: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:55.231: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:57.204: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain2e57 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a9c03619bcd3f48459f262d10b229cdb. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:36:59.358: INFO: Deleting namespace e2e-tests-service-lb-xhhsn

• [SLOW TEST:127.835 seconds]
Ensure LoadBalancer
/home/prow/go/src/sigs.k8s.io/cloud-provider-azure/tests/e2e/network/ensureloadbalancer.go:46
  should support multiple external services sharing one preset public IP address
... skipping 5 lines ...
Jul 16 08:37:11.681: INFO: Creating a kubernetes client
Jul 16 08:37:11.682: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:37:11.683: INFO: Creating a test namespace
Jul 16 08:37:11.793: INFO: Creating a kubernetes client
Jul 16 08:37:11.793: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:37:12.189: INFO: Creating deployment servicelb-test
Jul 16 08:37:12.529: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:37:22.630: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:37:32.631: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:37:42.631: INFO: Exposure successfully, get external ip: 51.11.51.151
Jul 16 08:37:42.631: INFO: Successfully created LoadBalancer service1 in namespace e2e-tests-service-lb-l4tnm with IP 51.11.51.151
Jul 16 08:37:42.860: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:37:52.963: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:38:02.961: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:38:12.963: INFO: Exposure successfully, get external ip: 51.11.51.151
Jul 16 08:38:12.963: INFO: Successfully created LoadBalancer service2 in namespace e2e-tests-service-lb-l4tnm with IP 51.11.51.151
STEP: Deleting one service and check if the other service works well
Jul 16 08:38:13.192: INFO: Exposure successfully, get external ip: 51.11.51.151
STEP: Deleting all services
STEP: Checking if the public IP has been deleted
... skipping 23 lines ...
Jul 16 08:39:32.612: INFO: Creating a kubernetes client
Jul 16 08:39:32.612: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:39:32.613: INFO: Creating a test namespace
Jul 16 08:39:32.718: INFO: Creating a kubernetes client
Jul 16 08:39:32.718: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:39:33.115: INFO: Creating deployment servicelb-test
Jul 16 08:39:33.449: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:39:43.550: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:39:53.552: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:40:03.554: INFO: Exposure successfully, get external ip: 10.240.0.8
Jul 16 08:40:03.554: INFO: Successfully created LoadBalancer service1 in namespace e2e-tests-service-lb-j8n2v with IP 10.240.0.8
Jul 16 08:40:03.775: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:40:13.880: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:40:23.876: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:40:33.881: INFO: Exposure successfully, get external ip: 10.240.0.8
Jul 16 08:40:33.881: INFO: Successfully created LoadBalancer service2 in namespace e2e-tests-service-lb-j8n2v with IP 10.240.0.8
STEP: Cleaning up
Jul 16 08:40:34.252: INFO: Deleting namespace e2e-tests-service-lb-j8n2v

• [SLOW TEST:115.946 seconds]
... skipping 124 lines ...
Jul 16 08:43:12.047: INFO: Creating a kubernetes client
Jul 16 08:43:12.047: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
STEP: Create service
Jul 16 08:43:12.446: INFO: Creating service annotation-test in namespace e2e-tests-service-d8w9s
Jul 16 08:43:12.561: INFO: Successfully created LoadBalancer service annotation-test in namespace e2e-tests-service-d8w9s
Jul 16 08:43:12.561: INFO: Waiting service to expose...
Jul 16 08:43:12.662: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:43:22.764: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:43:32.766: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:43:42.764: INFO: Exposure successfully, get external ip: 51.104.229.154
STEP: Validating External domain name
Jul 16 08:43:42.765: INFO: Get domain name: annotation-test92998732-52c9-45ac-a001-34a76c9cdb23.uksouth.cloudapp.azure.com
Jul 16 08:44:12.767: INFO: Received the following error when validating http://annotation-test92998732-52c9-45ac-a001-34a76c9cdb23.uksouth.cloudapp.azure.com:80: Get "http://annotation-test92998732-52c9-45ac-a001-34a76c9cdb23.uksouth.cloudapp.azure.com:80": dial tcp 51.104.229.154:80: i/o timeout
Jul 16 08:44:12.767: INFO: Retrying in 20 seconds
Jul 16 08:44:33.020: INFO: cleaning up test service annotation-test
Jul 16 08:44:33.150: INFO: Deleting service annotation-test in namespace e2e-tests-service-d8w9s
Jul 16 08:45:29.455: INFO: Deleting namespace e2e-tests-service-d8w9s

• [SLOW TEST:153.929 seconds]
... skipping 12 lines ...
Jul 16 08:45:45.969: INFO: Creating Azure clients
Jul 16 08:45:45.970: INFO: Creating a kubernetes client
Jul 16 08:45:45.970: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:45:46.365: INFO: Creating service annotation-test in namespace e2e-tests-service-h59d6
Jul 16 08:45:46.484: INFO: Successfully created LoadBalancer service annotation-test in namespace e2e-tests-service-h59d6
Jul 16 08:45:46.484: INFO: Waiting service to expose...
Jul 16 08:45:46.584: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:45:56.686: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:46:06.688: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:46:16.687: INFO: Exposure successfully, get external ip: 10.240.0.8
STEP: Validating whether the load balancer is internal
Jul 16 08:46:16.687: INFO: Validating external IP not be public and internal accessible
Jul 16 08:46:16.687: INFO: Create a front pod to connect to service
Jul 16 08:46:16.795: INFO: Call from the created pod
Jul 16 08:46:16.895: INFO: waiting for the pod succeeded
... skipping 25 lines ...
Jul 16 08:47:22.699: INFO: Getting virtual network list
STEP: Test subnet doesn't exist. Creating a new one...
Jul 16 08:47:24.566: INFO: creating a new subnet lb-subnet, 10.0.0.0/24
Jul 16 08:47:24.812: INFO: Creating service annotation-test in namespace e2e-tests-service-s72qm
Jul 16 08:47:24.931: INFO: Successfully created LoadBalancer service annotation-test in namespace e2e-tests-service-s72qm
Jul 16 08:47:24.931: INFO: Waiting service to expose...
Jul 16 08:47:25.031: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:47:35.133: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:47:45.133: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:47:55.133: INFO: Exposure successfully, get external ip: 10.0.0.4
Jul 16 08:47:55.133: INFO: Get External IP: 10.0.0.4
STEP: Validating external ip in target subnet
Jul 16 08:47:55.133: INFO: cleaning up test service annotation-test
Jul 16 08:47:55.254: INFO: Deleting service annotation-test in namespace e2e-tests-service-s72qm
Jul 16 08:48:27.454: INFO: cleaning up test subnet lb-subnet
... skipping 16 lines ...
Jul 16 08:48:44.603: INFO: Creating Azure clients
Jul 16 08:48:44.603: INFO: Creating a kubernetes client
Jul 16 08:48:44.603: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:48:44.998: INFO: Creating service annotation-test in namespace e2e-tests-service-shj6l
Jul 16 08:48:45.118: INFO: Successfully created LoadBalancer service annotation-test in namespace e2e-tests-service-shj6l
Jul 16 08:48:45.118: INFO: Waiting service to expose...
Jul 16 08:48:45.219: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:48:55.320: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:49:05.321: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:49:15.321: INFO: Exposure successfully, get external ip: 51.132.249.223
Jul 16 08:49:15.321: INFO: Getting public IPs in the resourceGroup kubetest-qhgwh1no
Jul 16 08:49:15.720: INFO: Getting public IP frontend configuration ID
Jul 16 08:49:15.720: INFO: Successfully obtained PIP front config id: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a114ae687b9dd46e6a44164eb1e1fdfb
Jul 16 08:49:15.720: INFO: Getting loadBalancer name from pipFrontendConfigurationID
Jul 16 08:49:15.720: INFO: Got loadBalancerName "kubetest-qhgwh1no"
... skipping 20 lines ...
STEP: resource group e2e-4de4 created
STEP: creating test PIP in the test resource group
Jul 16 08:50:30.813: INFO: Creating public IP resource named testPIP-986e
STEP: Creating service annotation-test in namespace e2e-tests-service-7jdpt
Jul 16 08:50:34.931: INFO: Successfully created LoadBalancer service annotation-test in namespace e2e-tests-service-7jdpt
STEP: Waiting service to expose...
Jul 16 08:50:35.031: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:50:45.132: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:50:55.134: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:51:05.134: INFO: Exposure successfully, get external ip: 51.132.255.175
Jul 16 08:51:05.134: INFO: Getting public IPs in the resourceGroup e2e-4de4
Jul 16 08:51:05.293: INFO: Getting public IP frontend configuration ID
Jul 16 08:51:05.293: INFO: Successfully obtained PIP front config id: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/a0dc576fdd7cd4027864828148401fe2
Jul 16 08:51:05.293: INFO: Getting loadBalancer name from pipFrontendConfigurationID
Jul 16 08:51:05.294: INFO: Got loadBalancerName "kubetest-qhgwh1no"
... skipping 22 lines ...
Jul 16 08:52:37.034: INFO: Creating a kubernetes client
Jul 16 08:52:37.034: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
STEP: Exposing two services with shared security rule
Jul 16 08:52:37.440: INFO: Creating service annotation-test in namespace e2e-tests-service-l6cms
Jul 16 08:52:37.559: INFO: Successfully created LoadBalancer service annotation-test in namespace e2e-tests-service-l6cms
Jul 16 08:52:37.559: INFO: Waiting service to expose...
Jul 16 08:52:37.666: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:52:47.771: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:52:57.771: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:53:07.771: INFO: Exposure successfully, get external ip: 20.49.184.219
Jul 16 08:53:07.771: INFO: Creating service annotation-test-share in namespace e2e-tests-service-l6cms
Jul 16 08:53:07.892: INFO: Successfully created LoadBalancer service annotation-test-share in namespace e2e-tests-service-l6cms
Jul 16 08:53:07.892: INFO: Waiting service to expose...
Jul 16 08:53:07.992: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:53:18.094: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:53:28.095: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:53:38.096: INFO: Exposure successfully, get external ip: 20.49.184.248
STEP: Validate shared security rule exists
Jul 16 08:53:38.096: INFO: Getting virtual network list
Jul 16 08:53:38.440: INFO: got sg list, length = 1
Jul 16 08:53:38.440: INFO: Found target security rule
STEP: Cleaning up
... skipping 16 lines ...
Jul 16 08:55:57.670: INFO: Creating deployment annotation-test
Jul 16 08:55:57.772: INFO: Creating Azure clients
Jul 16 08:55:57.772: INFO: Creating a kubernetes client
Jul 16 08:55:57.772: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
STEP: Creating a service with custom tags
STEP: Waiting service to expose...
Jul 16 08:55:58.397: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:56:08.499: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:56:18.499: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:56:28.499: INFO: Exposure successfully, get external ip: 20.49.186.189
STEP: Checking tags on the corresponding public IP
Jul 16 08:56:29.191: INFO: 
tags: map[a:0xc0006b06b0 c:0xc0006b06d0]
expectedTags: map[a:0xc000246ec0 c:0xc000246ed0]
STEP: Updating annotation and check tags again
... skipping 25 lines ...
Jul 16 08:57:40.645: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
STEP: Creating two test pips
Jul 16 08:57:41.065: INFO: Creating public IP resource named pip1
Jul 16 08:57:44.864: INFO: Creating public IP resource named pip2
STEP: Creating a service referring to the first pip
STEP: Waiting for the service to expose
Jul 16 08:57:47.484: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:57:57.586: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:58:07.586: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:58:17.587: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:58:27.586: INFO: Exposure successfully, get external ip: 20.49.188.156
STEP: Updating the service to refer to the second service
STEP: Waiting for service IP to be updated
Jul 16 08:58:27.909: INFO: Exposure successfully, get external ip: 20.49.188.156
Jul 16 08:58:38.014: INFO: Exposure successfully, get external ip: 20.49.188.156
Jul 16 08:58:48.013: INFO: Exposure successfully, get external ip: 20.49.188.156
... skipping 45 lines ...
Jul 16 09:01:04.086: INFO: Creating a kubernetes client
Jul 16 09:01:04.086: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 09:01:04.484: INFO: Creating deployment servicelb-test
Jul 16 09:01:04.591: INFO: Creating service servicelb-test in namespace e2e-tests-service-lb-h9b75
Jul 16 09:01:04.709: INFO: Successfully created LoadBalancer service servicelb-test in namespace e2e-tests-service-lb-h9b75
Jul 16 09:01:04.709: INFO: Waiting service to expose...
Jul 16 09:01:04.808: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 09:01:14.910: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 09:01:24.910: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 09:01:34.910: INFO: Exposure successfully, get external ip: 20.68.1.85
Jul 16 09:01:34.910: INFO: Getting public IPs in the resourceGroup kubetest-qhgwh1no
Jul 16 09:01:35.445: INFO: Getting public IP frontend configuration ID
Jul 16 09:01:35.445: INFO: Successfully obtained PIP front config id: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/af90faffe3b954e80b761d5dd065727e
Jul 16 09:01:35.445: INFO: Getting loadBalancer name from pipFrontendConfigurationID
Jul 16 09:01:35.445: INFO: Got loadBalancerName "kubetest-qhgwh1no"
... skipping 20 lines ...
Jul 16 09:02:38.154: INFO: Creating a kubernetes client
Jul 16 09:02:38.154: INFO: Kubernetes configuration file name: /root/tmp653352196/kubeconfig/kubeconfig.uksouth.json
Jul 16 09:02:38.555: INFO: Creating deployment servicelb-test
Jul 16 09:02:38.659: INFO: Creating service servicelb-test in namespace e2e-tests-service-lb-9zrh7
Jul 16 09:02:38.779: INFO: Successfully created LoadBalancer service servicelb-test in namespace e2e-tests-service-lb-9zrh7
Jul 16 09:02:38.779: INFO: Waiting service to expose...
Jul 16 09:02:38.878: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 09:02:48.984: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 09:02:58.981: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 09:03:08.983: INFO: Exposure successfully, get external ip: 20.68.4.46
Jul 16 09:03:08.983: INFO: Getting public IPs in the resourceGroup kubetest-qhgwh1no
Jul 16 09:03:09.481: INFO: Getting public IP frontend configuration ID
Jul 16 09:03:09.481: INFO: Successfully obtained PIP front config id: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-qhgwh1no/providers/Microsoft.Network/loadBalancers/kubetest-qhgwh1no/frontendIPConfigurations/ab63a5fc40d8640b9982e96aeb58fe81
Jul 16 09:03:09.481: INFO: Getting loadBalancer name from pipFrontendConfigurationID
Jul 16 09:03:09.481: INFO: Got loadBalancerName "kubetest-qhgwh1no"
... skipping 12 lines ...
  /home/prow/go/src/sigs.k8s.io/cloud-provider-azure/tests/e2e/network/standard_lb.go:139
------------------------------

JUnit report was created: /logs/artifacts/junit_01.xml

Ran 23 of 35 Specs in 3392.798 seconds
SUCCESS! -- 23 Passed | 0 Failed | 0 Pending | 12 Skipped
--- PASS: TestAzureTest (3392.80s)
PASS
ok  	sigs.k8s.io/cloud-provider-azure/tests/e2e	3392.826s
2021/07/16 09:04:12 process.go:155: Step 'make test-ccm-e2e' finished in 57m31.636780555s
2021/07/16 09:04:12 aksengine_helpers.go:425: downloading /root/tmp653352196/log-dump.sh from https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh
2021/07/16 09:04:12 util.go:68: curl https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh
... skipping 78 lines ...
ssh key file /root/.ssh/id_rsa does not exist. Exiting.
2021/07/16 09:05:38 process.go:155: Step 'bash -c /root/tmp653352196/win-ci-logs-collector.sh kubetest-qhgwh1no.uksouth.cloudapp.azure.com /root/tmp653352196 /root/.ssh/id_rsa' finished in 3.486144ms
2021/07/16 09:05:38 aksengine.go:1141: Deleting resource group: kubetest-qhgwh1no.
2021/07/16 09:13:18 process.go:96: Saved XML output to /logs/artifacts/junit_runner.xml.
2021/07/16 09:13:18 process.go:153: Running: bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}"
2021/07/16 09:13:19 process.go:155: Step 'bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}"' finished in 1.126317906s
2021/07/16 09:13:19 main.go:327: Something went wrong: encountered 1 errors: [error running log collection script /root/tmp653352196/log-dump.sh: error during bash -c /root/tmp653352196/log-dump.sh: exit status 1]
+ EXIT_VALUE=1
+ set +o xtrace
Cleaning up after docker in docker.
================================================================================
Cleaning up after docker
Stopping Docker: dockerProgram process in pidfile '/var/run/docker-ssd.pid', 1 process(es), refused to die.
... skipping 3 lines ...