This job view page is being replaced by Spyglass soon. Check out the new job view.
PRMirzaSikander: Not send availability zones as part of create for edge zones.
ResultFAILURE
Tests 1 failed / 37 succeeded
Started2021-07-16 07:57
Elapsed1h3m
Revision12f0aab9e350f1c8d62d85753eb8f3ac0dcb694d
Refs 708
job-versionv1.21.4-rc.0.1+1c29b7706e7642
kubetest-version
revisionv1.21.4-rc.0.1+1c29b7706e7642

Test Failures


kubetest DumpClusterLogs 2m44s

error running log collection script /root/tmp074957090/log-dump.sh: error during bash -c /root/tmp074957090/log-dump.sh: exit status 1
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 37 Passed Tests

Show 10 Skipped Tests

Error lines from build-log.txt

... skipping 222 lines ...
Jul 16 08:07:53.326: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:07:53.632: INFO: Creating deployment nsg-test
STEP: Creating a service and expose it
Jul 16 08:07:53.754: INFO: Creating service nsg-test in namespace e2e-tests-nsg-q87s6
Jul 16 08:07:53.873: INFO: Successfully created LoadBalancer service nsg-test in namespace e2e-tests-nsg-q87s6
Jul 16 08:07:53.873: INFO: Waiting service to expose...
Jul 16 08:07:53.973: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:08:04.076: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:08:14.082: INFO: Exposure successfully, get external ip: 20.90.228.203
STEP: Validating ip exists in Security Group
Jul 16 08:08:14.082: INFO: Getting virtual network list
Jul 16 08:08:14.609: INFO: got sg list, length = 1
Jul 16 08:08:14.609: INFO: Found target security rule
STEP: Validating network security group working
... skipping 24 lines ...
Jul 16 08:09:01.407: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:09:01.708: INFO: Creating deployment nsg-test
STEP: Creating service and wait it to expose
Jul 16 08:09:01.814: INFO: Creating service nsg-test in namespace e2e-tests-nsg-zhnhj
Jul 16 08:09:01.950: INFO: Successfully created LoadBalancer service nsg-test in namespace e2e-tests-nsg-zhnhj
Jul 16 08:09:01.950: INFO: Waiting service to expose...
Jul 16 08:09:02.050: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:09:12.153: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:09:22.152: INFO: Exposure successfully, get external ip: 20.90.227.144
STEP: Validating if the corresponding IP prefix existing in nsg
Jul 16 08:09:22.152: INFO: Getting virtual network list
Jul 16 08:09:22.414: INFO: got sg list, length = 1
Jul 16 08:09:22.520: INFO: Deleting namespace e2e-tests-nsg-zhnhj

... skipping 11 lines ...
Jul 16 08:10:00.833: INFO: Creating a test namespace
Jul 16 08:10:00.948: INFO: Creating a kubernetes client
Jul 16 08:10:00.948: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:10:01.249: INFO: Creating deployment nsg-test
STEP: Creating a test service with the deny rule annotation but without `service.Spec.LoadBalancerSourceRanges`
STEP: Waiting for the service to expose
Jul 16 08:10:01.597: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:10:11.702: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:10:21.699: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:10:31.699: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:10:41.699: INFO: Exposure successfully, get external ip: 10.240.0.6
STEP: Checking if there is a deny_all rule
Jul 16 08:10:41.699: INFO: Getting virtual network list
Jul 16 08:10:41.986: INFO: got sg list, length = 1
STEP: Deleting the service
Jul 16 08:10:42.107: INFO: Deleting service nsg-test in namespace e2e-tests-nsg-6mm8g
STEP: Creating a test service with the deny rule annotation and `service.Spec.LoadBalancerSourceRanges`
STEP: Waiting for the service to expose
Jul 16 08:11:26.529: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:11:36.633: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:11:46.634: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:11:56.634: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:12:06.631: INFO: Exposure successfully, get external ip: 10.240.0.6
STEP: Checking if there is a LoadBalancerSourceRanges rule
Jul 16 08:12:06.631: INFO: Getting virtual network list
Jul 16 08:12:06.786: INFO: got sg list, length = 1
STEP: Checking if there is a deny_all rule
Jul 16 08:12:06.891: INFO: Deleting namespace e2e-tests-nsg-6mm8g
... skipping 15 lines ...
Jul 16 08:13:01.408: INFO: Creating a kubernetes client
Jul 16 08:13:01.408: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
STEP: Create service
Jul 16 08:13:01.709: INFO: Creating service annotation-test in namespace e2e-tests-service-p6xrh
Jul 16 08:13:01.825: INFO: Successfully created LoadBalancer service annotation-test in namespace e2e-tests-service-p6xrh
Jul 16 08:13:01.825: INFO: Waiting service to expose...
Jul 16 08:13:01.926: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:13:12.028: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:13:22.028: INFO: Exposure successfully, get external ip: 20.77.128.174
STEP: Validating External domain name
Jul 16 08:13:22.030: INFO: Get domain name: annotation-testdb977b9b-3eb0-4bb8-bb5f-e454ea02afd2.uksouth.cloudapp.azure.com
Jul 16 08:13:52.030: INFO: Received the following error when validating http://annotation-testdb977b9b-3eb0-4bb8-bb5f-e454ea02afd2.uksouth.cloudapp.azure.com:80: Get "http://annotation-testdb977b9b-3eb0-4bb8-bb5f-e454ea02afd2.uksouth.cloudapp.azure.com:80": dial tcp 20.77.128.174:80: i/o timeout
Jul 16 08:13:52.030: INFO: Retrying in 20 seconds
Jul 16 08:14:12.322: INFO: cleaning up test service annotation-test
Jul 16 08:14:12.451: INFO: Deleting service annotation-test in namespace e2e-tests-service-p6xrh
Jul 16 08:14:36.757: INFO: Deleting namespace e2e-tests-service-p6xrh

• [SLOW TEST:111.867 seconds]
... skipping 12 lines ...
Jul 16 08:14:53.277: INFO: Creating Azure clients
Jul 16 08:14:53.277: INFO: Creating a kubernetes client
Jul 16 08:14:53.277: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:14:53.580: INFO: Creating service annotation-test in namespace e2e-tests-service-rgrxv
Jul 16 08:14:53.695: INFO: Successfully created LoadBalancer service annotation-test in namespace e2e-tests-service-rgrxv
Jul 16 08:14:53.695: INFO: Waiting service to expose...
Jul 16 08:14:53.807: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:15:03.910: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:15:13.909: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:15:23.912: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:15:33.910: INFO: Exposure successfully, get external ip: 10.240.0.6
STEP: Validating whether the load balancer is internal
Jul 16 08:15:33.910: INFO: Validating external IP not be public and internal accessible
Jul 16 08:15:33.910: INFO: Create a front pod to connect to service
Jul 16 08:15:34.033: INFO: Call from the created pod
Jul 16 08:15:34.134: INFO: waiting for the pod succeeded
... skipping 25 lines ...
Jul 16 08:16:43.845: INFO: Getting virtual network list
STEP: Test subnet doesn't exist. Creating a new one...
Jul 16 08:16:45.922: INFO: creating a new subnet lb-subnet, 10.0.0.0/24
Jul 16 08:16:46.192: INFO: Creating service annotation-test in namespace e2e-tests-service-d267c
Jul 16 08:16:46.305: INFO: Successfully created LoadBalancer service annotation-test in namespace e2e-tests-service-d267c
Jul 16 08:16:46.305: INFO: Waiting service to expose...
Jul 16 08:16:46.407: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:16:56.511: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:17:06.513: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:17:16.510: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:17:26.515: INFO: Exposure successfully, get external ip: 10.0.0.4
Jul 16 08:17:26.515: INFO: Get External IP: 10.0.0.4
STEP: Validating external ip in target subnet
Jul 16 08:17:26.515: INFO: cleaning up test service annotation-test
Jul 16 08:17:26.635: INFO: Deleting service annotation-test in namespace e2e-tests-service-d267c
Jul 16 08:18:08.840: INFO: cleaning up test subnet lb-subnet
... skipping 16 lines ...
Jul 16 08:18:21.846: INFO: Creating Azure clients
Jul 16 08:18:21.846: INFO: Creating a kubernetes client
Jul 16 08:18:21.846: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:18:22.158: INFO: Creating service annotation-test in namespace e2e-tests-service-9gjs4
Jul 16 08:18:22.276: INFO: Successfully created LoadBalancer service annotation-test in namespace e2e-tests-service-9gjs4
Jul 16 08:18:22.276: INFO: Waiting service to expose...
Jul 16 08:18:22.376: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:18:32.479: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:18:42.483: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:18:52.478: INFO: Exposure successfully, get external ip: 51.104.248.20
Jul 16 08:18:52.478: INFO: Getting public IPs in the resourceGroup kubetest-a1nrhve4
Jul 16 08:18:53.141: INFO: Getting public IP frontend configuration ID
Jul 16 08:18:53.141: INFO: Successfully obtained PIP front config id: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/a79a02711a54845b0bef3184e5360bae
Jul 16 08:18:53.141: INFO: Getting loadBalancer name from pipFrontendConfigurationID
Jul 16 08:18:53.141: INFO: Got loadBalancerName "kubetest-a1nrhve4"
... skipping 20 lines ...
STEP: resource group e2e-739d created
STEP: creating test PIP in the test resource group
Jul 16 08:19:25.889: INFO: Creating public IP resource named testPIP-ba1b
STEP: Creating service annotation-test in namespace e2e-tests-service-b6xqr
Jul 16 08:19:30.265: INFO: Successfully created LoadBalancer service annotation-test in namespace e2e-tests-service-b6xqr
STEP: Waiting service to expose...
Jul 16 08:19:30.366: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:19:40.469: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:19:50.470: INFO: Exposure successfully, get external ip: 51.141.236.178
Jul 16 08:19:50.470: INFO: Getting public IPs in the resourceGroup e2e-739d
Jul 16 08:19:50.621: INFO: Getting public IP frontend configuration ID
Jul 16 08:19:50.621: INFO: Successfully obtained PIP front config id: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/a3eb9f05f8b1c4c979f34cdb5ced0b93
Jul 16 08:19:50.621: INFO: Getting loadBalancer name from pipFrontendConfigurationID
Jul 16 08:19:50.621: INFO: Got loadBalancerName "kubetest-a1nrhve4"
... skipping 22 lines ...
Jul 16 08:20:59.443: INFO: Creating a kubernetes client
Jul 16 08:20:59.443: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
STEP: Exposing two services with shared security rule
Jul 16 08:20:59.744: INFO: Creating service annotation-test in namespace e2e-tests-service-7sc96
Jul 16 08:20:59.857: INFO: Successfully created LoadBalancer service annotation-test in namespace e2e-tests-service-7sc96
Jul 16 08:20:59.857: INFO: Waiting service to expose...
Jul 16 08:20:59.958: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:21:10.060: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:21:20.061: INFO: Exposure successfully, get external ip: 51.132.168.156
Jul 16 08:21:20.061: INFO: Creating service annotation-test-share in namespace e2e-tests-service-7sc96
Jul 16 08:21:20.178: INFO: Successfully created LoadBalancer service annotation-test-share in namespace e2e-tests-service-7sc96
Jul 16 08:21:20.178: INFO: Waiting service to expose...
Jul 16 08:21:20.278: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:21:30.381: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:21:40.380: INFO: Exposure successfully, get external ip: 20.77.129.93
STEP: Validate shared security rule exists
Jul 16 08:21:40.380: INFO: Getting virtual network list
Jul 16 08:21:40.746: INFO: got sg list, length = 1
Jul 16 08:21:40.746: INFO: Found target security rule
STEP: Cleaning up
... skipping 16 lines ...
Jul 16 08:23:07.911: INFO: Creating deployment annotation-test
Jul 16 08:23:08.017: INFO: Creating Azure clients
Jul 16 08:23:08.017: INFO: Creating a kubernetes client
Jul 16 08:23:08.017: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
STEP: Creating a service with custom tags
STEP: Waiting service to expose...
Jul 16 08:23:08.553: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:23:18.656: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:23:28.656: INFO: Exposure successfully, get external ip: 20.77.129.93
STEP: Checking tags on the corresponding public IP
Jul 16 08:23:29.737: INFO: 
tags: map[a:0xc000618250 c:0xc000618270]
expectedTags: map[a:0xc000208ee0 c:0xc000208ef0]
STEP: Updating annotation and check tags again
... skipping 25 lines ...
Jul 16 08:24:19.334: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
STEP: Creating two test pips
Jul 16 08:24:19.642: INFO: Creating public IP resource named pip1
Jul 16 08:24:23.002: INFO: Creating public IP resource named pip2
STEP: Creating a service referring to the first pip
STEP: Waiting for the service to expose
Jul 16 08:24:25.013: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:24:35.115: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:24:45.116: INFO: Exposure successfully, get external ip: 51.145.96.96
STEP: Updating the service to refer to the second service
STEP: Waiting for service IP to be updated
Jul 16 08:24:45.425: INFO: Exposure successfully, get external ip: 51.145.96.96
Jul 16 08:24:55.527: INFO: Exposure successfully, get external ip: 51.145.96.96
Jul 16 08:25:05.529: INFO: Exposure successfully, get external ip: 51.145.51.10
... skipping 69 lines ...
Jul 16 08:26:56.740: INFO: Creating a kubernetes client
Jul 16 08:26:56.740: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:26:57.041: INFO: Creating deployment servicelb-test
Jul 16 08:26:57.147: INFO: Creating service servicelb-test in namespace e2e-tests-service-lb-65wkf
Jul 16 08:26:57.266: INFO: Successfully created LoadBalancer service servicelb-test in namespace e2e-tests-service-lb-65wkf
Jul 16 08:26:57.266: INFO: Waiting service to expose...
Jul 16 08:26:57.367: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:27:07.469: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:27:17.472: INFO: Exposure successfully, get external ip: 51.104.248.20
Jul 16 08:27:17.472: INFO: Getting public IPs in the resourceGroup kubetest-a1nrhve4
Jul 16 08:27:17.868: INFO: Getting public IP frontend configuration ID
Jul 16 08:27:17.868: INFO: Successfully obtained PIP front config id: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/a1ad2b828c17643c3ba9fc58d1c96ddf
Jul 16 08:27:17.868: INFO: Getting loadBalancer name from pipFrontendConfigurationID
Jul 16 08:27:17.868: INFO: Got loadBalancerName "kubetest-a1nrhve4"
... skipping 18 lines ...
Jul 16 08:28:04.983: INFO: Creating a kubernetes client
Jul 16 08:28:04.983: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:28:05.284: INFO: Creating deployment servicelb-test
Jul 16 08:28:05.393: INFO: Creating service servicelb-test in namespace e2e-tests-service-lb-gwdn5
Jul 16 08:28:05.513: INFO: Successfully created LoadBalancer service servicelb-test in namespace e2e-tests-service-lb-gwdn5
Jul 16 08:28:05.513: INFO: Waiting service to expose...
Jul 16 08:28:05.614: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:28:15.717: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:28:25.721: INFO: Exposure successfully, get external ip: 51.104.248.20
Jul 16 08:28:25.721: INFO: Getting public IPs in the resourceGroup kubetest-a1nrhve4
Jul 16 08:28:26.007: INFO: Getting public IP frontend configuration ID
Jul 16 08:28:26.007: INFO: Successfully obtained PIP front config id: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/a700b7cca86924c98b6342527944d32c
Jul 16 08:28:26.007: INFO: Getting loadBalancer name from pipFrontendConfigurationID
Jul 16 08:28:26.008: INFO: Got loadBalancerName "kubetest-a1nrhve4"
... skipping 82 lines ...
Jul 16 08:29:46.838: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:29:47.138: INFO: Creating deployment servicelb-test
Jul 16 08:29:47.376: INFO: Successfully created LoadBalancer service servicelb-test in namespace e2e-tests-service-lb-q69qf
Jul 16 08:29:47.376: INFO: Creating public IP resource named service-lb-public-none-IP39b7
Jul 16 08:29:53.451: INFO: PIP to 51.104.21.231
STEP: Waiting for exposure of the original service without assigned lb IP
Jul 16 08:29:53.551: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:30:03.654: INFO: Exposure successfully, get external ip: 51.104.248.20
STEP: Updating service to bound to specific public IP
Jul 16 08:30:03.654: INFO: will update IP to 51.104.21.231
Jul 16 08:30:03.961: INFO: still unmatched external IP, retry it in 10s seconds
Jul 16 08:30:14.063: INFO: still unmatched external IP, retry it in 10s seconds
Jul 16 08:30:24.064: INFO: still unmatched external IP, retry it in 10s seconds
Jul 16 08:30:34.063: INFO: Exposure successfully
STEP: Cleaning up
Jul 16 08:30:34.182: INFO: Deleting public IP resource named service-lb-public-none-IP39b7 in resource group kubetest-a1nrhve4
Jul 16 08:30:34.636: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/publicIPAddresses/service-lb-public-none-IP39b7 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/a2206ad4d3e9e431fa58a7f61cb88789. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:30:37.022: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/publicIPAddresses/service-lb-public-none-IP39b7 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/a2206ad4d3e9e431fa58a7f61cb88789. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:30:40.534: INFO: Deleting namespace e2e-tests-service-lb-q69qf

• [SLOW TEST:70.127 seconds]
Ensure LoadBalancer
/home/prow/go/src/sigs.k8s.io/cloud-provider-azure/tests/e2e/network/ensureloadbalancer.go:46
  should support assigning to specific IP when updating public service
... skipping 8 lines ...
Jul 16 08:30:56.965: INFO: Creating a kubernetes client
Jul 16 08:30:56.966: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:30:57.266: INFO: Creating deployment servicelb-test
Jul 16 08:30:57.375: INFO: Getting virtual network list
Jul 16 08:31:01.737: INFO: Successfully created LoadBalancer service servicelb-test in namespace e2e-tests-service-lb-kthmz
STEP: Waiting for exposure of internal service with specific IP
Jul 16 08:31:01.837: INFO: Fail to get ingress, retry it in 10s seconds
Jul 16 08:31:11.942: INFO: Fail to get ingress, retry it in 10s seconds
Jul 16 08:31:21.940: INFO: Fail to get ingress, retry it in 10s seconds
Jul 16 08:31:31.943: INFO: Fail to get ingress, retry it in 10s seconds
Jul 16 08:31:41.943: INFO: Exposure successfully
Jul 16 08:31:42.144: INFO: Events list:
Jul 16 08:31:42.144: INFO: 0. {{ } {servicelb-test-76bf687774-8cj2w.1692383eb2b635a5  e2e-tests-service-lb-kthmz  728ca98b-bee3-47a1-bd4e-7e1a161dc365 10095 0 2021-07-16 08:30:57 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-scheduler Update events.k8s.io/v1 2021-07-16 08:30:57 +0000 UTC FieldsV1 {"f:action":{},"f:eventTime":{},"f:note":{},"f:reason":{},"f:regarding":{"f:apiVersion":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:reportingController":{},"f:reportingInstance":{},"f:type":{}} }]} {Pod e2e-tests-service-lb-kthmz servicelb-test-76bf687774-8cj2w eeb3f06f-1d0f-4411-999d-c108f3a214a0 v1 10084 } Scheduled Successfully assigned e2e-tests-service-lb-kthmz/servicelb-test-76bf687774-8cj2w to k8s-agentpool1-36955466-vmss000001 { } 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC 0 Normal 2021-07-16 08:30:57.371578 +0000 UTC nil Binding nil default-scheduler default-scheduler-k8s-master-36955466-0}
Jul 16 08:31:42.145: INFO: 1. {{ } {servicelb-test-76bf687774-8cj2w.1692383eefd4409e  e2e-tests-service-lb-kthmz  8db01652-fa87-4886-9deb-61577ab9a5f0 10117 0 2021-07-16 08:30:58 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2021-07-16 08:30:58 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{"f:apiVersion":{},"f:fieldPath":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]} {Pod e2e-tests-service-lb-kthmz servicelb-test-76bf687774-8cj2w eeb3f06f-1d0f-4411-999d-c108f3a214a0 v1 10087 spec.containers{test-app}} Pulling Pulling image "nginx:1.15" {kubelet k8s-agentpool1-36955466-vmss000001} 2021-07-16 08:30:58 +0000 UTC 2021-07-16 08:30:58 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }
Jul 16 08:31:42.145: INFO: 2. {{ } {servicelb-test-76bf687774-8cj2w.1692383f643db0dd  e2e-tests-service-lb-kthmz  619bdf3f-1031-432a-9bf1-449787f82944 10135 0 2021-07-16 08:31:00 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2021-07-16 08:31:00 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{"f:apiVersion":{},"f:fieldPath":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]} {Pod e2e-tests-service-lb-kthmz servicelb-test-76bf687774-8cj2w eeb3f06f-1d0f-4411-999d-c108f3a214a0 v1 10087 spec.containers{test-app}} Pulled Successfully pulled image "nginx:1.15" in 1.953057971s {kubelet k8s-agentpool1-36955466-vmss000001} 2021-07-16 08:31:00 +0000 UTC 2021-07-16 08:31:00 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }
Jul 16 08:31:42.145: INFO: 3. {{ } {servicelb-test-76bf687774-8cj2w.1692383f678634ad  e2e-tests-service-lb-kthmz  6a2a6326-2282-4106-b9c8-cfc5b9c4ac40 10138 0 2021-07-16 08:31:00 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2021-07-16 08:31:00 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{"f:apiVersion":{},"f:fieldPath":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]} {Pod e2e-tests-service-lb-kthmz servicelb-test-76bf687774-8cj2w eeb3f06f-1d0f-4411-999d-c108f3a214a0 v1 10087 spec.containers{test-app}} Created Created container test-app {kubelet k8s-agentpool1-36955466-vmss000001} 2021-07-16 08:31:00 +0000 UTC 2021-07-16 08:31:00 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }
... skipping 50 lines ...
Jul 16 08:32:38.641: INFO: Creating a kubernetes client
Jul 16 08:32:38.641: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:32:38.941: INFO: Creating deployment servicelb-test
Jul 16 08:32:39.169: INFO: Successfully created LoadBalancer service servicelb-test in namespace e2e-tests-service-lb-tp4m4
Jul 16 08:32:39.169: INFO: Creating public IP resource named service-lb-internal-none-public-IP5694
STEP: Waiting for exposure of the original service without assigned lb private IP
Jul 16 08:32:42.992: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:32:53.095: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:33:03.093: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:33:13.094: INFO: Exposure successfully, get external ip: 10.240.0.6
Jul 16 08:33:13.295: INFO: Events list:
Jul 16 08:33:13.295: INFO: 0. {{ } {servicelb-test-76bf687774-6w6xt.1692385662dbc2ad  e2e-tests-service-lb-tp4m4  f6d19206-8579-4267-9e7b-10c247277f19 10702 0 2021-07-16 08:32:39 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-scheduler Update events.k8s.io/v1 2021-07-16 08:32:39 +0000 UTC FieldsV1 {"f:action":{},"f:eventTime":{},"f:note":{},"f:reason":{},"f:regarding":{"f:apiVersion":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:reportingController":{},"f:reportingInstance":{},"f:type":{}} }]} {Pod e2e-tests-service-lb-tp4m4 servicelb-test-76bf687774-6w6xt d8e4e700-e5c3-41b5-9e5b-c17263675628 v1 10687 } Scheduled Successfully assigned e2e-tests-service-lb-tp4m4/servicelb-test-76bf687774-6w6xt to k8s-agentpool1-36955466-vmss000001 { } 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC 0 Normal 2021-07-16 08:32:39.111077 +0000 UTC nil Binding nil default-scheduler default-scheduler-k8s-master-36955466-0}
Jul 16 08:33:13.295: INFO: 1. {{ } {servicelb-test-76bf687774-6w6xt.16923856986f224b  e2e-tests-service-lb-tp4m4  b31e3117-100e-4bca-b944-4ce15c711df6 10715 0 2021-07-16 08:32:40 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2021-07-16 08:32:40 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{"f:apiVersion":{},"f:fieldPath":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]} {Pod e2e-tests-service-lb-tp4m4 servicelb-test-76bf687774-6w6xt d8e4e700-e5c3-41b5-9e5b-c17263675628 v1 10694 spec.containers{test-app}} Pulling Pulling image "nginx:1.15" {kubelet k8s-agentpool1-36955466-vmss000001} 2021-07-16 08:32:40 +0000 UTC 2021-07-16 08:32:40 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }
Jul 16 08:33:13.295: INFO: 2. {{ } {servicelb-test-76bf687774-6w6xt.16923856d592f944  e2e-tests-service-lb-tp4m4  08c080a7-f695-4ce9-a4d9-f8c205cb32af 10726 0 2021-07-16 08:32:41 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2021-07-16 08:32:41 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{"f:apiVersion":{},"f:fieldPath":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]} {Pod e2e-tests-service-lb-tp4m4 servicelb-test-76bf687774-6w6xt d8e4e700-e5c3-41b5-9e5b-c17263675628 v1 10694 spec.containers{test-app}} Pulled Successfully pulled image "nginx:1.15" in 1.025745869s {kubelet k8s-agentpool1-36955466-vmss000001} 2021-07-16 08:32:41 +0000 UTC 2021-07-16 08:32:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }
Jul 16 08:33:13.295: INFO: 3. {{ } {servicelb-test-76bf687774-6w6xt.16923856da60c796  e2e-tests-service-lb-tp4m4  23752a3f-5fb8-4636-82e2-1d6e133dddaf 10728 0 2021-07-16 08:32:41 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2021-07-16 08:32:41 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{"f:apiVersion":{},"f:fieldPath":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]} {Pod e2e-tests-service-lb-tp4m4 servicelb-test-76bf687774-6w6xt d8e4e700-e5c3-41b5-9e5b-c17263675628 v1 10694 spec.containers{test-app}} Created Created container test-app {kubelet k8s-agentpool1-36955466-vmss000001} 2021-07-16 08:32:41 +0000 UTC 2021-07-16 08:32:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }
... skipping 34 lines ...
Jul 16 08:33:43.717: INFO: still unmatched external IP, retry it in 10s seconds
Jul 16 08:33:53.718: INFO: still unmatched external IP, retry it in 10s seconds
Jul 16 08:34:03.718: INFO: still unmatched external IP, retry it in 10s seconds
Jul 16 08:34:13.719: INFO: Exposure successfully
STEP: Cleaning up
Jul 16 08:34:13.837: INFO: Deleting public IP resource named service-lb-internal-none-public-IP5694 in resource group kubetest-a1nrhve4
Jul 16 08:34:14.744: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/publicIPAddresses/service-lb-internal-none-public-IP5694 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/a74385fa2cd544d4ca2ef5c7e38ec5c4. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:34:17.142: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/publicIPAddresses/service-lb-internal-none-public-IP5694 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/a74385fa2cd544d4ca2ef5c7e38ec5c4. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:34:19.289: INFO: Deleting namespace e2e-tests-service-lb-tp4m4

• [SLOW TEST:113.060 seconds]
Ensure LoadBalancer
/home/prow/go/src/sigs.k8s.io/cloud-provider-azure/tests/e2e/network/ensureloadbalancer.go:46
  should support updating an internal service to a public service with assigned IP
... skipping 8 lines ...
Jul 16 08:34:31.701: INFO: Creating a kubernetes client
Jul 16 08:34:31.701: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:34:32.002: INFO: Creating deployment servicelb-test
Jul 16 08:34:32.107: INFO: Creating public IP resource named service-lb-public-remain1c52
Jul 16 08:34:36.069: INFO: Successfully created LoadBalancer service servicelb-test in namespace e2e-tests-service-lb-wppvn
STEP: Waiting for exposure of the original service with assigned lb private IP
Jul 16 08:34:36.169: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:34:46.272: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:34:56.272: INFO: Exposure successfully, get external ip: 51.11.12.134
STEP: Update without changing the service and wait for a while
Jul 16 08:34:56.272: INFO: External IP is now 51.11.12.134
Jul 16 08:34:56.372: INFO: testlabel1c52
Jul 16 08:34:56.579: INFO: External IP is still 51.11.12.134
Jul 16 08:35:06.681: INFO: External IP is still 51.11.12.134
... skipping 56 lines ...
Jul 16 08:44:36.682: INFO: External IP is still 51.11.12.134
Jul 16 08:44:46.683: INFO: External IP is still 51.11.12.134
Jul 16 08:44:56.682: INFO: External IP is still 51.11.12.134
Jul 16 08:44:56.789: INFO: External IP is still 51.11.12.134
STEP: Cleaning up
Jul 16 08:44:56.909: INFO: Deleting public IP resource named service-lb-public-remain1c52 in resource group kubetest-a1nrhve4
Jul 16 08:44:57.753: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain1c52 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/a0406eb4b81c14fdb8a3613d3618fcef. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:45:00.134: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain1c52 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/a0406eb4b81c14fdb8a3613d3618fcef. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:45:02.282: INFO: Deleting namespace e2e-tests-service-lb-wppvn

• [SLOW TEST:646.991 seconds]
Ensure LoadBalancer
/home/prow/go/src/sigs.k8s.io/cloud-provider-azure/tests/e2e/network/ensureloadbalancer.go:46
  should have no operation since no change in service when update [Slow]
... skipping 6 lines ...
Jul 16 08:45:18.587: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:45:18.589: INFO: Creating a test namespace
Jul 16 08:45:18.692: INFO: Creating a kubernetes client
Jul 16 08:45:18.692: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:45:18.992: INFO: Creating deployment servicelb-test
Jul 16 08:45:19.097: INFO: Creating public IP resource named service-lb-public-remain6008
Jul 16 08:45:23.498: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:45:33.604: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:45:43.600: INFO: Exposure successfully, get external ip: 51.11.15.120
Jul 16 08:45:43.600: INFO: Successfully created LoadBalancer service1 in namespace e2e-tests-service-lb-rf46p with IP 51.11.15.120
Jul 16 08:45:43.819: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:45:53.921: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:46:03.922: INFO: Exposure successfully, get external ip: 51.11.15.120
Jul 16 08:46:03.922: INFO: Successfully created LoadBalancer service2 in namespace e2e-tests-service-lb-rf46p with IP 51.11.15.120
STEP: Cleaning up
Jul 16 08:46:04.161: INFO: Deleting public IP resource named service-lb-public-remain6008 in resource group kubetest-a1nrhve4
Jul 16 08:46:04.572: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain6008 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/abbd64e1dd6ee4ac79fb50475c1fcb45. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:46:07.007: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain6008 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/abbd64e1dd6ee4ac79fb50475c1fcb45. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:46:08.961: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain6008 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/abbd64e1dd6ee4ac79fb50475c1fcb45. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:46:10.970: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain6008 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/abbd64e1dd6ee4ac79fb50475c1fcb45. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:46:13.282: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain6008 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/abbd64e1dd6ee4ac79fb50475c1fcb45. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:46:14.958: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain6008 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/abbd64e1dd6ee4ac79fb50475c1fcb45. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:46:16.967: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain6008 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/abbd64e1dd6ee4ac79fb50475c1fcb45. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:46:18.963: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain6008 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/abbd64e1dd6ee4ac79fb50475c1fcb45. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:46:20.989: INFO: error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/publicIPAddresses/service-lb-public-remain6008 can not be deleted since it is still allocated to resource /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/abbd64e1dd6ee4ac79fb50475c1fcb45. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[], will retry soon
Jul 16 08:46:23.108: INFO: Deleting namespace e2e-tests-service-lb-rf46p

• [SLOW TEST:76.851 seconds]
Ensure LoadBalancer
/home/prow/go/src/sigs.k8s.io/cloud-provider-azure/tests/e2e/network/ensureloadbalancer.go:46
  should support multiple external services sharing one preset public IP address
... skipping 5 lines ...
Jul 16 08:46:35.438: INFO: Creating a kubernetes client
Jul 16 08:46:35.438: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:46:35.440: INFO: Creating a test namespace
Jul 16 08:46:35.546: INFO: Creating a kubernetes client
Jul 16 08:46:35.546: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:46:35.848: INFO: Creating deployment servicelb-test
Jul 16 08:46:36.175: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:46:46.278: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:46:56.277: INFO: Exposure successfully, get external ip: 20.77.131.229
Jul 16 08:46:56.277: INFO: Successfully created LoadBalancer service1 in namespace e2e-tests-service-lb-htzzn with IP 20.77.131.229
Jul 16 08:46:56.495: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:47:06.597: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:47:16.596: INFO: Exposure successfully, get external ip: 20.77.131.229
Jul 16 08:47:16.596: INFO: Successfully created LoadBalancer service2 in namespace e2e-tests-service-lb-htzzn with IP 20.77.131.229
STEP: Deleting one service and check if the other service works well
Jul 16 08:47:16.815: INFO: Exposure successfully, get external ip: 20.77.131.229
STEP: Deleting all services
STEP: Checking if the public IP has been deleted
... skipping 19 lines ...
Jul 16 08:48:13.902: INFO: Creating a kubernetes client
Jul 16 08:48:13.902: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:48:13.904: INFO: Creating a test namespace
Jul 16 08:48:14.007: INFO: Creating a kubernetes client
Jul 16 08:48:14.007: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:48:14.309: INFO: Creating deployment servicelb-test
Jul 16 08:48:14.647: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:48:24.751: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:48:34.750: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:48:44.750: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:48:54.749: INFO: Exposure successfully, get external ip: 10.240.0.6
Jul 16 08:48:54.749: INFO: Successfully created LoadBalancer service1 in namespace e2e-tests-service-lb-m2n5d with IP 10.240.0.6
Jul 16 08:48:54.971: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:49:05.074: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:49:15.075: INFO: Exposure successfully, get external ip: 10.240.0.6
Jul 16 08:49:15.075: INFO: Successfully created LoadBalancer service2 in namespace e2e-tests-service-lb-m2n5d with IP 10.240.0.6
STEP: Cleaning up
Jul 16 08:49:15.425: INFO: Deleting namespace e2e-tests-service-lb-m2n5d

• [SLOW TEST:111.831 seconds]
... skipping 10 lines ...
Jul 16 08:50:05.735: INFO: Creating a test namespace
Jul 16 08:50:05.839: INFO: Creating a kubernetes client
Jul 16 08:50:05.839: INFO: Kubernetes configuration file name: /root/tmp074957090/kubeconfig/kubeconfig.uksouth.json
Jul 16 08:50:06.138: INFO: Creating deployment servicelb-test
STEP: Checking the number of the node pools
STEP: Creating a service to trigger the LB reconcile
Jul 16 08:50:06.663: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:50:16.766: INFO: Fail to find ingress, retry it in 10 seconds
Jul 16 08:50:26.769: INFO: Exposure successfully, get external ip: 51.132.43.136
STEP: Checking the initial node number in the LB backend pool
Jul 16 08:50:26.769: INFO: Getting public IPs in the resourceGroup kubetest-a1nrhve4
Jul 16 08:50:27.316: INFO: Getting public IP frontend configuration ID
Jul 16 08:50:27.316: INFO: Successfully obtained PIP front config id: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-a1nrhve4/providers/Microsoft.Network/loadBalancers/kubetest-a1nrhve4/frontendIPConfigurations/a81e7ec7d6adc45ecbfc075df18bf0b0
Jul 16 08:50:27.316: INFO: Getting loadBalancer name from pipFrontendConfigurationID
... skipping 68 lines ...
  /home/prow/go/src/sigs.k8s.io/cloud-provider-azure/tests/e2e/network/service_annotations.go:495
------------------------------

JUnit report was created: /logs/artifacts/junit_01.xml

Ran 25 of 35 Specs in 2632.299 seconds
SUCCESS! -- 25 Passed | 0 Failed | 0 Pending | 10 Skipped
--- PASS: TestAzureTest (2632.30s)
PASS
ok  	sigs.k8s.io/cloud-provider-azure/tests/e2e	2632.338s
2021/07/16 08:51:45 process.go:155: Step 'make test-ccm-e2e' finished in 44m57.892616945s
2021/07/16 08:51:45 aksengine_helpers.go:425: downloading /root/tmp074957090/log-dump.sh from https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh
2021/07/16 08:51:45 util.go:68: curl https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh
... skipping 101 lines ...
Dumping kube-apiserver-k8s-master-36955466-2_kube-system_kube-apiserver-ef9731b6ee7268e7ceaae4df108ee1962d4b94b4e7b6023b9e845e181257848d.log
Dumping kube-controller-manager-k8s-master-36955466-2_kube-system_kube-controller-manager-559313f25ad86e8b65421c4a26ec0f3f527c04ec177296adc4c0dd002d10fa3a.log
Dumping kube-proxy-9jws9_kube-system_kube-proxy-ea539a697fef397270c7e630322e3b0c857ab8edbf4530285d98926c5ca4f120.log
Dumping kube-scheduler-k8s-master-36955466-2_kube-system_kube-scheduler-8373bd0edd9af054e2138db84eb789b66a32011bc27658fb3940a901b99f8532.log
Dumping kubelet.log
Dumping docker.log
Unable to connect to the server: local error: tls: bad record MAC
Uninstalling log-dump-daemonset.yaml via kubectl
daemonset.apps "log-dump-node" deleted
2021/07/16 08:54:30 process.go:155: Step 'bash -c /root/tmp074957090/log-dump.sh' finished in 2m44.567397392s
2021/07/16 08:54:30 aksengine_helpers.go:425: downloading /root/tmp074957090/win-ci-logs-collector.sh from https://raw.githubusercontent.com/kubernetes-sigs/windows-testing/master/scripts/win-ci-logs-collector.sh
2021/07/16 08:54:30 util.go:68: curl https://raw.githubusercontent.com/kubernetes-sigs/windows-testing/master/scripts/win-ci-logs-collector.sh
2021/07/16 08:54:30 process.go:153: Running: chmod +x /root/tmp074957090/win-ci-logs-collector.sh
... skipping 2 lines ...
ssh key file /root/.ssh/id_rsa does not exist. Exiting.
2021/07/16 08:54:30 process.go:155: Step 'bash -c /root/tmp074957090/win-ci-logs-collector.sh kubetest-a1nrhve4.uksouth.cloudapp.azure.com /root/tmp074957090 /root/.ssh/id_rsa' finished in 3.80468ms
2021/07/16 08:54:30 aksengine.go:1141: Deleting resource group: kubetest-a1nrhve4.
2021/07/16 09:00:09 process.go:96: Saved XML output to /logs/artifacts/junit_runner.xml.
2021/07/16 09:00:09 process.go:153: Running: bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}"
2021/07/16 09:00:10 process.go:155: Step 'bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}"' finished in 1.194493606s
2021/07/16 09:00:10 main.go:327: Something went wrong: encountered 1 errors: [error running log collection script /root/tmp074957090/log-dump.sh: error during bash -c /root/tmp074957090/log-dump.sh: exit status 1]
+ EXIT_VALUE=1
+ set +o xtrace
Cleaning up after docker in docker.
================================================================================
Cleaning up after docker
Stopping Docker: dockerProgram process in pidfile '/var/run/docker-ssd.pid', 1 process(es), refused to die.
... skipping 3 lines ...