kubernetes-sigs/azurefile-csi-driver PR #1143 pull-azurefile-csi-driver-e2e-windows-containerd #1607313066934931456
Recent runs || View in Spyglass
Recent runs || View in Spyglass
| PR | cvvz: fix: support privateendpoint with latest cloud-provider-azure |
| Result | FAILURE |
| Tests | 1 failed / 14 succeeded |
| Started | |
| Elapsed | 1h17m |
| Revision | 5ea5ebb853be1ee4917876d251029b0681e01a80 |
| Refs |
1143 |
| job-version | v1.27.0-alpha.0.629+685d639cb5f04c |
| kubetest-version | v20221223-736a4da5ba |
| revision | v1.27.0-alpha.0.629+685d639cb5f04c |
error during make e2e-test: exit status 2
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest Build
kubetest Check APIReachability
kubetest Deferred TearDown
kubetest DumpClusterLogs
kubetest GetDeployer
kubetest IsUp
kubetest Prepare
kubetest TearDown
kubetest TearDown Previous
kubetest Timeout
kubetest Up
kubetest kubectl version
kubetest list nodes
kubetest test setup
... skipping 106 lines ... 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 11345 100 11345 0 0 181k 0 --:--:-- --:--:-- --:--:-- 181k Downloading https://get.helm.sh/helm-v3.10.3-linux-amd64.tar.gz Verifying checksum... Done. Preparing to install helm into /usr/local/bin helm installed into /usr/local/bin/helm docker pull k8sprow.azurecr.io/azurefile-csi:e2e-e60d471c240439ae2631573c35838e7a7cb19931 || make container-all push-manifest Error response from daemon: manifest for k8sprow.azurecr.io/azurefile-csi:e2e-e60d471c240439ae2631573c35838e7a7cb19931 not found: manifest unknown: manifest tagged by "e2e-e60d471c240439ae2631573c35838e7a7cb19931" is not found make[2]: Entering directory '/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver' CGO_ENABLED=0 GOOS=windows go build -a -ldflags "-X sigs.k8s.io/azurefile-csi-driver/pkg/azurefile.driverVersion=e2e-e60d471c240439ae2631573c35838e7a7cb19931 -X sigs.k8s.io/azurefile-csi-driver/pkg/azurefile.gitCommit=e60d471c240439ae2631573c35838e7a7cb19931 -X sigs.k8s.io/azurefile-csi-driver/pkg/azurefile.buildDate=2022-12-26T09:59:59Z -s -w -extldflags '-static'" -mod vendor -o _output/amd64/azurefileplugin.exe ./pkg/azurefileplugin docker buildx rm container-builder || true ERROR: no builder "container-builder" found docker buildx create --use --name=container-builder container-builder # enable qemu for arm64 build # https://github.com/docker/buildx/issues/464#issuecomment-741507760 docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-aarch64 Unable to find image 'tonistiigi/binfmt:latest' locally ... skipping 2533 lines ... image: "mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.6.0" args: - "-csi-address=$(ADDRESS)" - "-v=2" - "-leader-election" - "--leader-election-namespace=kube-system" - '-handle-volume-inuse-error=false' - '-timeout=120s' - '-feature-gates=RecoverVolumeExpansionFailure=true' env: - name: ADDRESS value: /csi/csi.sock imagePullPolicy: IfNotPresent ... skipping 190 lines ... Dec 26 10:13:25.152: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-snstt] to have phase Bound Dec 26 10:13:25.255: INFO: PersistentVolumeClaim pvc-snstt found and phase=Bound (103.186482ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:13:25.255[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:13:25.359[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:13:25.461[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:13:25.462[0m [1mSTEP:[0m checking that the pods command exits with an error [38;5;243m12/26/22 10:13:25.566[0m Dec 26 10:13:25.567: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-lm744" in namespace "azurefile-8081" to be "Error status code" Dec 26 10:13:25.669: INFO: Pod "azurefile-volume-tester-lm744": Phase="Pending", Reason="", readiness=false. Elapsed: 102.282326ms Dec 26 10:13:27.773: INFO: Pod "azurefile-volume-tester-lm744": Phase="Pending", Reason="", readiness=false. Elapsed: 2.206445535s Dec 26 10:13:29.787: INFO: Pod "azurefile-volume-tester-lm744": Phase="Pending", Reason="", readiness=false. Elapsed: 4.220642738s Dec 26 10:13:31.774: INFO: Pod "azurefile-volume-tester-lm744": Phase="Pending", Reason="", readiness=false. Elapsed: 6.207410098s Dec 26 10:13:33.774: INFO: Pod "azurefile-volume-tester-lm744": Phase="Pending", Reason="", readiness=false. Elapsed: 8.207281823s Dec 26 10:13:35.772: INFO: Pod "azurefile-volume-tester-lm744": Phase="Pending", Reason="", readiness=false. Elapsed: 10.205503966s Dec 26 10:13:37.772: INFO: Pod "azurefile-volume-tester-lm744": Phase="Pending", Reason="", readiness=false. Elapsed: 12.20542906s Dec 26 10:13:39.773: INFO: Pod "azurefile-volume-tester-lm744": Phase="Running", Reason="", readiness=true. Elapsed: 14.206195683s Dec 26 10:13:41.773: INFO: Pod "azurefile-volume-tester-lm744": Phase="Running", Reason="", readiness=true. Elapsed: 16.205844943s Dec 26 10:13:43.773: INFO: Pod "azurefile-volume-tester-lm744": Phase="Running", Reason="", readiness=true. Elapsed: 18.206792011s Dec 26 10:13:45.772: INFO: Pod "azurefile-volume-tester-lm744": Phase="Running", Reason="", readiness=true. Elapsed: 20.205455058s Dec 26 10:13:47.773: INFO: Pod "azurefile-volume-tester-lm744": Phase="Running", Reason="", readiness=true. Elapsed: 22.2062163s Dec 26 10:13:49.773: INFO: Pod "azurefile-volume-tester-lm744": Phase="Running", Reason="", readiness=false. Elapsed: 24.206628988s Dec 26 10:13:51.780: INFO: Pod "azurefile-volume-tester-lm744": Phase="Failed", Reason="", readiness=false. Elapsed: 26.212897956s [1mSTEP:[0m Saw pod failure [38;5;243m12/26/22 10:13:51.78[0m Dec 26 10:13:51.780: INFO: Pod "azurefile-volume-tester-lm744" satisfied condition "Error status code" [1mSTEP:[0m checking that pod logs contain expected message [38;5;243m12/26/22 10:13:51.78[0m Dec 26 10:13:51.918: INFO: deleting Pod "azurefile-8081"/"azurefile-volume-tester-lm744" Dec 26 10:13:52.033: INFO: Pod azurefile-volume-tester-lm744 has the following logs: Out-File : Access to the path 'C:\mnt\test-1\data.txt' is denied. At line:1 char:22 + ... cho 'hello world' | Out-File -FilePath C:\mnt\test-1\data.txt; Get-Co ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ... skipping 73 lines ... Dec 26 10:13:25.152: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-snstt] to have phase Bound Dec 26 10:13:25.255: INFO: PersistentVolumeClaim pvc-snstt found and phase=Bound (103.186482ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:13:25.255[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:13:25.359[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:13:25.461[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:13:25.462[0m [1mSTEP:[0m checking that the pods command exits with an error [38;5;243m12/26/22 10:13:25.566[0m Dec 26 10:13:25.567: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-lm744" in namespace "azurefile-8081" to be "Error status code" Dec 26 10:13:25.669: INFO: Pod "azurefile-volume-tester-lm744": Phase="Pending", Reason="", readiness=false. Elapsed: 102.282326ms Dec 26 10:13:27.773: INFO: Pod "azurefile-volume-tester-lm744": Phase="Pending", Reason="", readiness=false. Elapsed: 2.206445535s Dec 26 10:13:29.787: INFO: Pod "azurefile-volume-tester-lm744": Phase="Pending", Reason="", readiness=false. Elapsed: 4.220642738s Dec 26 10:13:31.774: INFO: Pod "azurefile-volume-tester-lm744": Phase="Pending", Reason="", readiness=false. Elapsed: 6.207410098s Dec 26 10:13:33.774: INFO: Pod "azurefile-volume-tester-lm744": Phase="Pending", Reason="", readiness=false. Elapsed: 8.207281823s Dec 26 10:13:35.772: INFO: Pod "azurefile-volume-tester-lm744": Phase="Pending", Reason="", readiness=false. Elapsed: 10.205503966s Dec 26 10:13:37.772: INFO: Pod "azurefile-volume-tester-lm744": Phase="Pending", Reason="", readiness=false. Elapsed: 12.20542906s Dec 26 10:13:39.773: INFO: Pod "azurefile-volume-tester-lm744": Phase="Running", Reason="", readiness=true. Elapsed: 14.206195683s Dec 26 10:13:41.773: INFO: Pod "azurefile-volume-tester-lm744": Phase="Running", Reason="", readiness=true. Elapsed: 16.205844943s Dec 26 10:13:43.773: INFO: Pod "azurefile-volume-tester-lm744": Phase="Running", Reason="", readiness=true. Elapsed: 18.206792011s Dec 26 10:13:45.772: INFO: Pod "azurefile-volume-tester-lm744": Phase="Running", Reason="", readiness=true. Elapsed: 20.205455058s Dec 26 10:13:47.773: INFO: Pod "azurefile-volume-tester-lm744": Phase="Running", Reason="", readiness=true. Elapsed: 22.2062163s Dec 26 10:13:49.773: INFO: Pod "azurefile-volume-tester-lm744": Phase="Running", Reason="", readiness=false. Elapsed: 24.206628988s Dec 26 10:13:51.780: INFO: Pod "azurefile-volume-tester-lm744": Phase="Failed", Reason="", readiness=false. Elapsed: 26.212897956s [1mSTEP:[0m Saw pod failure [38;5;243m12/26/22 10:13:51.78[0m Dec 26 10:13:51.780: INFO: Pod "azurefile-volume-tester-lm744" satisfied condition "Error status code" [1mSTEP:[0m checking that pod logs contain expected message [38;5;243m12/26/22 10:13:51.78[0m Dec 26 10:13:51.918: INFO: deleting Pod "azurefile-8081"/"azurefile-volume-tester-lm744" Dec 26 10:13:52.033: INFO: Pod azurefile-volume-tester-lm744 has the following logs: Out-File : Access to the path 'C:\mnt\test-1\data.txt' is denied. At line:1 char:22 + ... cho 'hello world' | Out-File -FilePath C:\mnt\test-1\data.txt; Get-Co ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ... skipping 49 lines ... Dec 26 10:13:56.491: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-shjdm] to have phase Bound Dec 26 10:13:56.592: INFO: PersistentVolumeClaim pvc-shjdm found and phase=Bound (101.877563ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:13:56.592[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:13:56.695[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:13:56.797[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:13:56.797[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:13:56.903[0m Dec 26 10:13:56.903: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-h2dfn" in namespace "azurefile-2540" to be "Succeeded or Failed" Dec 26 10:13:57.005: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Pending", Reason="", readiness=false. Elapsed: 102.181551ms Dec 26 10:13:59.109: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205805232s Dec 26 10:14:01.108: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Pending", Reason="", readiness=false. Elapsed: 4.20565968s Dec 26 10:14:03.109: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Pending", Reason="", readiness=false. Elapsed: 6.206140912s Dec 26 10:14:05.109: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Pending", Reason="", readiness=false. Elapsed: 8.206055876s Dec 26 10:14:07.108: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Pending", Reason="", readiness=false. Elapsed: 10.205546578s ... skipping 5 lines ... Dec 26 10:14:19.109: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Running", Reason="", readiness=true. Elapsed: 22.206150308s Dec 26 10:14:21.110: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Running", Reason="", readiness=true. Elapsed: 24.207253294s Dec 26 10:14:23.108: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Running", Reason="", readiness=true. Elapsed: 26.205523767s Dec 26 10:14:25.109: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Running", Reason="", readiness=false. Elapsed: 28.206552538s Dec 26 10:14:27.109: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Succeeded", Reason="", readiness=false. Elapsed: 30.206012221s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:14:27.109[0m Dec 26 10:14:27.109: INFO: Pod "azurefile-volume-tester-h2dfn" satisfied condition "Succeeded or Failed" [1mSTEP:[0m setting up the PV [38;5;243m12/26/22 10:14:27.109[0m [1mSTEP:[0m creating a PV [38;5;243m12/26/22 10:14:27.109[0m [1mSTEP:[0m setting up the PVC [38;5;243m12/26/22 10:14:27.213[0m [1mSTEP:[0m creating a PVC [38;5;243m12/26/22 10:14:27.213[0m [1mSTEP:[0m waiting for PVC to be in phase "Bound" [38;5;243m12/26/22 10:14:27.322[0m Dec 26 10:14:27.322: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-4hf64] to have phase Bound Dec 26 10:14:27.425: INFO: PersistentVolumeClaim pvc-4hf64 found and phase=Bound (102.585086ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:14:27.425[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:14:27.528[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:14:27.632[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:14:27.632[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:14:27.738[0m Dec 26 10:14:27.738: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-plfmj" in namespace "azurefile-2540" to be "Succeeded or Failed" Dec 26 10:14:27.842: INFO: Pod "azurefile-volume-tester-plfmj": Phase="Pending", Reason="", readiness=false. Elapsed: 103.268164ms Dec 26 10:14:29.946: INFO: Pod "azurefile-volume-tester-plfmj": Phase="Pending", Reason="", readiness=false. Elapsed: 2.207385735s Dec 26 10:14:31.947: INFO: Pod "azurefile-volume-tester-plfmj": Phase="Pending", Reason="", readiness=false. Elapsed: 4.208123844s Dec 26 10:14:33.947: INFO: Pod "azurefile-volume-tester-plfmj": Phase="Running", Reason="", readiness=true. Elapsed: 6.208175411s Dec 26 10:14:35.949: INFO: Pod "azurefile-volume-tester-plfmj": Phase="Running", Reason="", readiness=true. Elapsed: 8.210766747s Dec 26 10:14:37.948: INFO: Pod "azurefile-volume-tester-plfmj": Phase="Running", Reason="", readiness=false. Elapsed: 10.209502792s Dec 26 10:14:39.947: INFO: Pod "azurefile-volume-tester-plfmj": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.209039978s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:14:39.948[0m Dec 26 10:14:39.948: INFO: Pod "azurefile-volume-tester-plfmj" satisfied condition "Succeeded or Failed" [1mSTEP:[0m setting up the PV [38;5;243m12/26/22 10:14:39.948[0m [1mSTEP:[0m creating a PV [38;5;243m12/26/22 10:14:39.948[0m [1mSTEP:[0m setting up the PVC [38;5;243m12/26/22 10:14:40.052[0m [1mSTEP:[0m creating a PVC [38;5;243m12/26/22 10:14:40.052[0m [1mSTEP:[0m waiting for PVC to be in phase "Bound" [38;5;243m12/26/22 10:14:40.155[0m Dec 26 10:14:40.156: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-ccjcr] to have phase Bound Dec 26 10:14:40.261: INFO: PersistentVolumeClaim pvc-ccjcr found and phase=Bound (105.008497ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:14:40.261[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:14:40.364[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:14:40.467[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:14:40.467[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:14:40.574[0m Dec 26 10:14:40.574: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-tsvsw" in namespace "azurefile-2540" to be "Succeeded or Failed" Dec 26 10:14:40.678: INFO: Pod "azurefile-volume-tester-tsvsw": Phase="Pending", Reason="", readiness=false. Elapsed: 103.585249ms Dec 26 10:14:42.781: INFO: Pod "azurefile-volume-tester-tsvsw": Phase="Pending", Reason="", readiness=false. Elapsed: 2.207354455s Dec 26 10:14:44.782: INFO: Pod "azurefile-volume-tester-tsvsw": Phase="Pending", Reason="", readiness=false. Elapsed: 4.207965887s Dec 26 10:14:46.781: INFO: Pod "azurefile-volume-tester-tsvsw": Phase="Running", Reason="", readiness=true. Elapsed: 6.206976332s Dec 26 10:14:48.782: INFO: Pod "azurefile-volume-tester-tsvsw": Phase="Running", Reason="", readiness=true. Elapsed: 8.207879008s Dec 26 10:14:50.782: INFO: Pod "azurefile-volume-tester-tsvsw": Phase="Running", Reason="", readiness=false. Elapsed: 10.207679892s Dec 26 10:14:52.781: INFO: Pod "azurefile-volume-tester-tsvsw": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.207469802s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:14:52.781[0m Dec 26 10:14:52.782: INFO: Pod "azurefile-volume-tester-tsvsw" satisfied condition "Succeeded or Failed" [1mSTEP:[0m setting up the PV [38;5;243m12/26/22 10:14:52.782[0m [1mSTEP:[0m creating a PV [38;5;243m12/26/22 10:14:52.782[0m [1mSTEP:[0m setting up the PVC [38;5;243m12/26/22 10:14:52.887[0m [1mSTEP:[0m creating a PVC [38;5;243m12/26/22 10:14:52.888[0m [1mSTEP:[0m waiting for PVC to be in phase "Bound" [38;5;243m12/26/22 10:14:52.991[0m Dec 26 10:14:52.992: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-8rdvn] to have phase Bound Dec 26 10:14:53.094: INFO: PersistentVolumeClaim pvc-8rdvn found and phase=Bound (102.242032ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:14:53.094[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:14:53.197[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:14:53.299[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:14:53.3[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:14:53.406[0m Dec 26 10:14:53.406: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-hh9vx" in namespace "azurefile-2540" to be "Succeeded or Failed" Dec 26 10:14:53.509: INFO: Pod "azurefile-volume-tester-hh9vx": Phase="Pending", Reason="", readiness=false. Elapsed: 102.398107ms Dec 26 10:14:55.613: INFO: Pod "azurefile-volume-tester-hh9vx": Phase="Pending", Reason="", readiness=false. Elapsed: 2.20645501s Dec 26 10:14:57.612: INFO: Pod "azurefile-volume-tester-hh9vx": Phase="Pending", Reason="", readiness=false. Elapsed: 4.20599697s Dec 26 10:14:59.615: INFO: Pod "azurefile-volume-tester-hh9vx": Phase="Running", Reason="", readiness=true. Elapsed: 6.2082746s Dec 26 10:15:01.612: INFO: Pod "azurefile-volume-tester-hh9vx": Phase="Running", Reason="", readiness=true. Elapsed: 8.205665948s Dec 26 10:15:03.613: INFO: Pod "azurefile-volume-tester-hh9vx": Phase="Running", Reason="", readiness=false. Elapsed: 10.206805438s Dec 26 10:15:05.614: INFO: Pod "azurefile-volume-tester-hh9vx": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.207461114s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:15:05.614[0m Dec 26 10:15:05.614: INFO: Pod "azurefile-volume-tester-hh9vx" satisfied condition "Succeeded or Failed" [1mSTEP:[0m setting up the PV [38;5;243m12/26/22 10:15:05.614[0m [1mSTEP:[0m creating a PV [38;5;243m12/26/22 10:15:05.614[0m [1mSTEP:[0m setting up the PVC [38;5;243m12/26/22 10:15:05.72[0m [1mSTEP:[0m creating a PVC [38;5;243m12/26/22 10:15:05.72[0m [1mSTEP:[0m waiting for PVC to be in phase "Bound" [38;5;243m12/26/22 10:15:05.826[0m Dec 26 10:15:05.826: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-6h9lw] to have phase Bound Dec 26 10:15:05.930: INFO: PersistentVolumeClaim pvc-6h9lw found and phase=Bound (103.325206ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:15:05.93[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:15:06.033[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:15:06.137[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:15:06.137[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:15:06.242[0m Dec 26 10:15:06.242: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-9znrk" in namespace "azurefile-2540" to be "Succeeded or Failed" Dec 26 10:15:06.344: INFO: Pod "azurefile-volume-tester-9znrk": Phase="Pending", Reason="", readiness=false. Elapsed: 102.054443ms Dec 26 10:15:08.447: INFO: Pod "azurefile-volume-tester-9znrk": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205540619s Dec 26 10:15:10.449: INFO: Pod "azurefile-volume-tester-9znrk": Phase="Pending", Reason="", readiness=false. Elapsed: 4.207044094s Dec 26 10:15:12.513: INFO: Pod "azurefile-volume-tester-9znrk": Phase="Running", Reason="", readiness=true. Elapsed: 6.27098656s Dec 26 10:15:14.447: INFO: Pod "azurefile-volume-tester-9znrk": Phase="Running", Reason="", readiness=true. Elapsed: 8.205343901s Dec 26 10:15:16.465: INFO: Pod "azurefile-volume-tester-9znrk": Phase="Running", Reason="", readiness=false. Elapsed: 10.223047969s Dec 26 10:15:18.448: INFO: Pod "azurefile-volume-tester-9znrk": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.206568838s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:15:18.449[0m Dec 26 10:15:18.449: INFO: Pod "azurefile-volume-tester-9znrk" satisfied condition "Succeeded or Failed" [1mSTEP:[0m setting up the PV [38;5;243m12/26/22 10:15:18.449[0m [1mSTEP:[0m creating a PV [38;5;243m12/26/22 10:15:18.449[0m [1mSTEP:[0m setting up the PVC [38;5;243m12/26/22 10:15:18.554[0m [1mSTEP:[0m creating a PVC [38;5;243m12/26/22 10:15:18.554[0m [1mSTEP:[0m waiting for PVC to be in phase "Bound" [38;5;243m12/26/22 10:15:18.659[0m Dec 26 10:15:18.659: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-4wg6f] to have phase Bound Dec 26 10:15:18.764: INFO: PersistentVolumeClaim pvc-4wg6f found and phase=Bound (105.327638ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:15:18.764[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:15:18.867[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:15:18.97[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:15:18.97[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:15:19.076[0m Dec 26 10:15:19.076: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-tz9cm" in namespace "azurefile-2540" to be "Succeeded or Failed" Dec 26 10:15:19.179: INFO: Pod "azurefile-volume-tester-tz9cm": Phase="Pending", Reason="", readiness=false. Elapsed: 102.783059ms Dec 26 10:15:21.283: INFO: Pod "azurefile-volume-tester-tz9cm": Phase="Pending", Reason="", readiness=false. Elapsed: 2.207315255s Dec 26 10:15:23.282: INFO: Pod "azurefile-volume-tester-tz9cm": Phase="Pending", Reason="", readiness=false. Elapsed: 4.206689732s Dec 26 10:15:25.282: INFO: Pod "azurefile-volume-tester-tz9cm": Phase="Running", Reason="", readiness=true. Elapsed: 6.206627214s Dec 26 10:15:27.282: INFO: Pod "azurefile-volume-tester-tz9cm": Phase="Running", Reason="", readiness=true. Elapsed: 8.206267479s Dec 26 10:15:29.282: INFO: Pod "azurefile-volume-tester-tz9cm": Phase="Running", Reason="", readiness=false. Elapsed: 10.206749341s Dec 26 10:15:31.283: INFO: Pod "azurefile-volume-tester-tz9cm": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.20685296s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:15:31.283[0m Dec 26 10:15:31.283: INFO: Pod "azurefile-volume-tester-tz9cm" satisfied condition "Succeeded or Failed" Dec 26 10:15:31.283: INFO: deleting Pod "azurefile-2540"/"azurefile-volume-tester-tz9cm" Dec 26 10:15:31.415: INFO: Pod azurefile-volume-tester-tz9cm has the following logs: hello world [1mSTEP:[0m Deleting pod azurefile-volume-tester-tz9cm in namespace azurefile-2540 [38;5;243m12/26/22 10:15:31.415[0m Dec 26 10:15:31.535: INFO: deleting PVC "azurefile-2540"/"pvc-4wg6f" Dec 26 10:15:31.535: INFO: Deleting PersistentVolumeClaim "pvc-4wg6f" ... skipping 94 lines ... Dec 26 10:13:56.491: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-shjdm] to have phase Bound Dec 26 10:13:56.592: INFO: PersistentVolumeClaim pvc-shjdm found and phase=Bound (101.877563ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:13:56.592[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:13:56.695[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:13:56.797[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:13:56.797[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:13:56.903[0m Dec 26 10:13:56.903: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-h2dfn" in namespace "azurefile-2540" to be "Succeeded or Failed" Dec 26 10:13:57.005: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Pending", Reason="", readiness=false. Elapsed: 102.181551ms Dec 26 10:13:59.109: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205805232s Dec 26 10:14:01.108: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Pending", Reason="", readiness=false. Elapsed: 4.20565968s Dec 26 10:14:03.109: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Pending", Reason="", readiness=false. Elapsed: 6.206140912s Dec 26 10:14:05.109: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Pending", Reason="", readiness=false. Elapsed: 8.206055876s Dec 26 10:14:07.108: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Pending", Reason="", readiness=false. Elapsed: 10.205546578s ... skipping 5 lines ... Dec 26 10:14:19.109: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Running", Reason="", readiness=true. Elapsed: 22.206150308s Dec 26 10:14:21.110: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Running", Reason="", readiness=true. Elapsed: 24.207253294s Dec 26 10:14:23.108: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Running", Reason="", readiness=true. Elapsed: 26.205523767s Dec 26 10:14:25.109: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Running", Reason="", readiness=false. Elapsed: 28.206552538s Dec 26 10:14:27.109: INFO: Pod "azurefile-volume-tester-h2dfn": Phase="Succeeded", Reason="", readiness=false. Elapsed: 30.206012221s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:14:27.109[0m Dec 26 10:14:27.109: INFO: Pod "azurefile-volume-tester-h2dfn" satisfied condition "Succeeded or Failed" [1mSTEP:[0m setting up the PV [38;5;243m12/26/22 10:14:27.109[0m [1mSTEP:[0m creating a PV [38;5;243m12/26/22 10:14:27.109[0m [1mSTEP:[0m setting up the PVC [38;5;243m12/26/22 10:14:27.213[0m [1mSTEP:[0m creating a PVC [38;5;243m12/26/22 10:14:27.213[0m [1mSTEP:[0m waiting for PVC to be in phase "Bound" [38;5;243m12/26/22 10:14:27.322[0m Dec 26 10:14:27.322: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-4hf64] to have phase Bound Dec 26 10:14:27.425: INFO: PersistentVolumeClaim pvc-4hf64 found and phase=Bound (102.585086ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:14:27.425[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:14:27.528[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:14:27.632[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:14:27.632[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:14:27.738[0m Dec 26 10:14:27.738: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-plfmj" in namespace "azurefile-2540" to be "Succeeded or Failed" Dec 26 10:14:27.842: INFO: Pod "azurefile-volume-tester-plfmj": Phase="Pending", Reason="", readiness=false. Elapsed: 103.268164ms Dec 26 10:14:29.946: INFO: Pod "azurefile-volume-tester-plfmj": Phase="Pending", Reason="", readiness=false. Elapsed: 2.207385735s Dec 26 10:14:31.947: INFO: Pod "azurefile-volume-tester-plfmj": Phase="Pending", Reason="", readiness=false. Elapsed: 4.208123844s Dec 26 10:14:33.947: INFO: Pod "azurefile-volume-tester-plfmj": Phase="Running", Reason="", readiness=true. Elapsed: 6.208175411s Dec 26 10:14:35.949: INFO: Pod "azurefile-volume-tester-plfmj": Phase="Running", Reason="", readiness=true. Elapsed: 8.210766747s Dec 26 10:14:37.948: INFO: Pod "azurefile-volume-tester-plfmj": Phase="Running", Reason="", readiness=false. Elapsed: 10.209502792s Dec 26 10:14:39.947: INFO: Pod "azurefile-volume-tester-plfmj": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.209039978s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:14:39.948[0m Dec 26 10:14:39.948: INFO: Pod "azurefile-volume-tester-plfmj" satisfied condition "Succeeded or Failed" [1mSTEP:[0m setting up the PV [38;5;243m12/26/22 10:14:39.948[0m [1mSTEP:[0m creating a PV [38;5;243m12/26/22 10:14:39.948[0m [1mSTEP:[0m setting up the PVC [38;5;243m12/26/22 10:14:40.052[0m [1mSTEP:[0m creating a PVC [38;5;243m12/26/22 10:14:40.052[0m [1mSTEP:[0m waiting for PVC to be in phase "Bound" [38;5;243m12/26/22 10:14:40.155[0m Dec 26 10:14:40.156: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-ccjcr] to have phase Bound Dec 26 10:14:40.261: INFO: PersistentVolumeClaim pvc-ccjcr found and phase=Bound (105.008497ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:14:40.261[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:14:40.364[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:14:40.467[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:14:40.467[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:14:40.574[0m Dec 26 10:14:40.574: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-tsvsw" in namespace "azurefile-2540" to be "Succeeded or Failed" Dec 26 10:14:40.678: INFO: Pod "azurefile-volume-tester-tsvsw": Phase="Pending", Reason="", readiness=false. Elapsed: 103.585249ms Dec 26 10:14:42.781: INFO: Pod "azurefile-volume-tester-tsvsw": Phase="Pending", Reason="", readiness=false. Elapsed: 2.207354455s Dec 26 10:14:44.782: INFO: Pod "azurefile-volume-tester-tsvsw": Phase="Pending", Reason="", readiness=false. Elapsed: 4.207965887s Dec 26 10:14:46.781: INFO: Pod "azurefile-volume-tester-tsvsw": Phase="Running", Reason="", readiness=true. Elapsed: 6.206976332s Dec 26 10:14:48.782: INFO: Pod "azurefile-volume-tester-tsvsw": Phase="Running", Reason="", readiness=true. Elapsed: 8.207879008s Dec 26 10:14:50.782: INFO: Pod "azurefile-volume-tester-tsvsw": Phase="Running", Reason="", readiness=false. Elapsed: 10.207679892s Dec 26 10:14:52.781: INFO: Pod "azurefile-volume-tester-tsvsw": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.207469802s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:14:52.781[0m Dec 26 10:14:52.782: INFO: Pod "azurefile-volume-tester-tsvsw" satisfied condition "Succeeded or Failed" [1mSTEP:[0m setting up the PV [38;5;243m12/26/22 10:14:52.782[0m [1mSTEP:[0m creating a PV [38;5;243m12/26/22 10:14:52.782[0m [1mSTEP:[0m setting up the PVC [38;5;243m12/26/22 10:14:52.887[0m [1mSTEP:[0m creating a PVC [38;5;243m12/26/22 10:14:52.888[0m [1mSTEP:[0m waiting for PVC to be in phase "Bound" [38;5;243m12/26/22 10:14:52.991[0m Dec 26 10:14:52.992: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-8rdvn] to have phase Bound Dec 26 10:14:53.094: INFO: PersistentVolumeClaim pvc-8rdvn found and phase=Bound (102.242032ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:14:53.094[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:14:53.197[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:14:53.299[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:14:53.3[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:14:53.406[0m Dec 26 10:14:53.406: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-hh9vx" in namespace "azurefile-2540" to be "Succeeded or Failed" Dec 26 10:14:53.509: INFO: Pod "azurefile-volume-tester-hh9vx": Phase="Pending", Reason="", readiness=false. Elapsed: 102.398107ms Dec 26 10:14:55.613: INFO: Pod "azurefile-volume-tester-hh9vx": Phase="Pending", Reason="", readiness=false. Elapsed: 2.20645501s Dec 26 10:14:57.612: INFO: Pod "azurefile-volume-tester-hh9vx": Phase="Pending", Reason="", readiness=false. Elapsed: 4.20599697s Dec 26 10:14:59.615: INFO: Pod "azurefile-volume-tester-hh9vx": Phase="Running", Reason="", readiness=true. Elapsed: 6.2082746s Dec 26 10:15:01.612: INFO: Pod "azurefile-volume-tester-hh9vx": Phase="Running", Reason="", readiness=true. Elapsed: 8.205665948s Dec 26 10:15:03.613: INFO: Pod "azurefile-volume-tester-hh9vx": Phase="Running", Reason="", readiness=false. Elapsed: 10.206805438s Dec 26 10:15:05.614: INFO: Pod "azurefile-volume-tester-hh9vx": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.207461114s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:15:05.614[0m Dec 26 10:15:05.614: INFO: Pod "azurefile-volume-tester-hh9vx" satisfied condition "Succeeded or Failed" [1mSTEP:[0m setting up the PV [38;5;243m12/26/22 10:15:05.614[0m [1mSTEP:[0m creating a PV [38;5;243m12/26/22 10:15:05.614[0m [1mSTEP:[0m setting up the PVC [38;5;243m12/26/22 10:15:05.72[0m [1mSTEP:[0m creating a PVC [38;5;243m12/26/22 10:15:05.72[0m [1mSTEP:[0m waiting for PVC to be in phase "Bound" [38;5;243m12/26/22 10:15:05.826[0m Dec 26 10:15:05.826: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-6h9lw] to have phase Bound Dec 26 10:15:05.930: INFO: PersistentVolumeClaim pvc-6h9lw found and phase=Bound (103.325206ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:15:05.93[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:15:06.033[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:15:06.137[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:15:06.137[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:15:06.242[0m Dec 26 10:15:06.242: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-9znrk" in namespace "azurefile-2540" to be "Succeeded or Failed" Dec 26 10:15:06.344: INFO: Pod "azurefile-volume-tester-9znrk": Phase="Pending", Reason="", readiness=false. Elapsed: 102.054443ms Dec 26 10:15:08.447: INFO: Pod "azurefile-volume-tester-9znrk": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205540619s Dec 26 10:15:10.449: INFO: Pod "azurefile-volume-tester-9znrk": Phase="Pending", Reason="", readiness=false. Elapsed: 4.207044094s Dec 26 10:15:12.513: INFO: Pod "azurefile-volume-tester-9znrk": Phase="Running", Reason="", readiness=true. Elapsed: 6.27098656s Dec 26 10:15:14.447: INFO: Pod "azurefile-volume-tester-9znrk": Phase="Running", Reason="", readiness=true. Elapsed: 8.205343901s Dec 26 10:15:16.465: INFO: Pod "azurefile-volume-tester-9znrk": Phase="Running", Reason="", readiness=false. Elapsed: 10.223047969s Dec 26 10:15:18.448: INFO: Pod "azurefile-volume-tester-9znrk": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.206568838s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:15:18.449[0m Dec 26 10:15:18.449: INFO: Pod "azurefile-volume-tester-9znrk" satisfied condition "Succeeded or Failed" [1mSTEP:[0m setting up the PV [38;5;243m12/26/22 10:15:18.449[0m [1mSTEP:[0m creating a PV [38;5;243m12/26/22 10:15:18.449[0m [1mSTEP:[0m setting up the PVC [38;5;243m12/26/22 10:15:18.554[0m [1mSTEP:[0m creating a PVC [38;5;243m12/26/22 10:15:18.554[0m [1mSTEP:[0m waiting for PVC to be in phase "Bound" [38;5;243m12/26/22 10:15:18.659[0m Dec 26 10:15:18.659: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-4wg6f] to have phase Bound Dec 26 10:15:18.764: INFO: PersistentVolumeClaim pvc-4wg6f found and phase=Bound (105.327638ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:15:18.764[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:15:18.867[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:15:18.97[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:15:18.97[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:15:19.076[0m Dec 26 10:15:19.076: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-tz9cm" in namespace "azurefile-2540" to be "Succeeded or Failed" Dec 26 10:15:19.179: INFO: Pod "azurefile-volume-tester-tz9cm": Phase="Pending", Reason="", readiness=false. Elapsed: 102.783059ms Dec 26 10:15:21.283: INFO: Pod "azurefile-volume-tester-tz9cm": Phase="Pending", Reason="", readiness=false. Elapsed: 2.207315255s Dec 26 10:15:23.282: INFO: Pod "azurefile-volume-tester-tz9cm": Phase="Pending", Reason="", readiness=false. Elapsed: 4.206689732s Dec 26 10:15:25.282: INFO: Pod "azurefile-volume-tester-tz9cm": Phase="Running", Reason="", readiness=true. Elapsed: 6.206627214s Dec 26 10:15:27.282: INFO: Pod "azurefile-volume-tester-tz9cm": Phase="Running", Reason="", readiness=true. Elapsed: 8.206267479s Dec 26 10:15:29.282: INFO: Pod "azurefile-volume-tester-tz9cm": Phase="Running", Reason="", readiness=false. Elapsed: 10.206749341s Dec 26 10:15:31.283: INFO: Pod "azurefile-volume-tester-tz9cm": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.20685296s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:15:31.283[0m Dec 26 10:15:31.283: INFO: Pod "azurefile-volume-tester-tz9cm" satisfied condition "Succeeded or Failed" Dec 26 10:15:31.283: INFO: deleting Pod "azurefile-2540"/"azurefile-volume-tester-tz9cm" Dec 26 10:15:31.415: INFO: Pod azurefile-volume-tester-tz9cm has the following logs: hello world [1mSTEP:[0m Deleting pod azurefile-volume-tester-tz9cm in namespace azurefile-2540 [38;5;243m12/26/22 10:15:31.415[0m Dec 26 10:15:31.535: INFO: deleting PVC "azurefile-2540"/"pvc-4wg6f" Dec 26 10:15:31.535: INFO: Deleting PersistentVolumeClaim "pvc-4wg6f" ... skipping 174 lines ... Dec 26 10:15:44.815: INFO: PersistentVolumeClaim pvc-wl5ts found but phase is Pending instead of Bound. Dec 26 10:15:46.919: INFO: PersistentVolumeClaim pvc-wl5ts found and phase=Bound (2.206130798s) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:15:46.919[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:15:47.03[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:15:47.157[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:15:47.157[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:15:47.272[0m Dec 26 10:15:47.272: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-pgkls" in namespace "azurefile-5466" to be "Succeeded or Failed" Dec 26 10:15:47.387: INFO: Pod "azurefile-volume-tester-pgkls": Phase="Pending", Reason="", readiness=false. Elapsed: 114.435661ms Dec 26 10:15:49.491: INFO: Pod "azurefile-volume-tester-pgkls": Phase="Pending", Reason="", readiness=false. Elapsed: 2.21862697s Dec 26 10:15:51.491: INFO: Pod "azurefile-volume-tester-pgkls": Phase="Pending", Reason="", readiness=false. Elapsed: 4.21883016s Dec 26 10:15:53.491: INFO: Pod "azurefile-volume-tester-pgkls": Phase="Running", Reason="", readiness=true. Elapsed: 6.218072611s Dec 26 10:15:55.492: INFO: Pod "azurefile-volume-tester-pgkls": Phase="Running", Reason="", readiness=true. Elapsed: 8.219594991s Dec 26 10:15:57.492: INFO: Pod "azurefile-volume-tester-pgkls": Phase="Running", Reason="", readiness=false. Elapsed: 10.219945641s Dec 26 10:15:59.492: INFO: Pod "azurefile-volume-tester-pgkls": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.219658116s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:15:59.492[0m Dec 26 10:15:59.492: INFO: Pod "azurefile-volume-tester-pgkls" satisfied condition "Succeeded or Failed" Dec 26 10:15:59.492: INFO: deleting Pod "azurefile-5466"/"azurefile-volume-tester-pgkls" Dec 26 10:15:59.599: INFO: Pod azurefile-volume-tester-pgkls has the following logs: hello world [1mSTEP:[0m Deleting pod azurefile-volume-tester-pgkls in namespace azurefile-5466 [38;5;243m12/26/22 10:15:59.599[0m Dec 26 10:15:59.720: INFO: deleting PVC "azurefile-5466"/"pvc-wl5ts" Dec 26 10:15:59.720: INFO: Deleting PersistentVolumeClaim "pvc-wl5ts" ... skipping 27 lines ... Dec 26 10:15:44.815: INFO: PersistentVolumeClaim pvc-wl5ts found but phase is Pending instead of Bound. Dec 26 10:15:46.919: INFO: PersistentVolumeClaim pvc-wl5ts found and phase=Bound (2.206130798s) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:15:46.919[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:15:47.03[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:15:47.157[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:15:47.157[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:15:47.272[0m Dec 26 10:15:47.272: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-pgkls" in namespace "azurefile-5466" to be "Succeeded or Failed" Dec 26 10:15:47.387: INFO: Pod "azurefile-volume-tester-pgkls": Phase="Pending", Reason="", readiness=false. Elapsed: 114.435661ms Dec 26 10:15:49.491: INFO: Pod "azurefile-volume-tester-pgkls": Phase="Pending", Reason="", readiness=false. Elapsed: 2.21862697s Dec 26 10:15:51.491: INFO: Pod "azurefile-volume-tester-pgkls": Phase="Pending", Reason="", readiness=false. Elapsed: 4.21883016s Dec 26 10:15:53.491: INFO: Pod "azurefile-volume-tester-pgkls": Phase="Running", Reason="", readiness=true. Elapsed: 6.218072611s Dec 26 10:15:55.492: INFO: Pod "azurefile-volume-tester-pgkls": Phase="Running", Reason="", readiness=true. Elapsed: 8.219594991s Dec 26 10:15:57.492: INFO: Pod "azurefile-volume-tester-pgkls": Phase="Running", Reason="", readiness=false. Elapsed: 10.219945641s Dec 26 10:15:59.492: INFO: Pod "azurefile-volume-tester-pgkls": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.219658116s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:15:59.492[0m Dec 26 10:15:59.492: INFO: Pod "azurefile-volume-tester-pgkls" satisfied condition "Succeeded or Failed" Dec 26 10:15:59.492: INFO: deleting Pod "azurefile-5466"/"azurefile-volume-tester-pgkls" Dec 26 10:15:59.599: INFO: Pod azurefile-volume-tester-pgkls has the following logs: hello world [1mSTEP:[0m Deleting pod azurefile-volume-tester-pgkls in namespace azurefile-5466 [38;5;243m12/26/22 10:15:59.599[0m Dec 26 10:15:59.720: INFO: deleting PVC "azurefile-5466"/"pvc-wl5ts" Dec 26 10:15:59.720: INFO: Deleting PersistentVolumeClaim "pvc-wl5ts" ... skipping 27 lines ... Dec 26 10:16:03.912: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-mpfw4] to have phase Bound Dec 26 10:16:04.020: INFO: PersistentVolumeClaim pvc-mpfw4 found and phase=Bound (107.992033ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:16:04.02[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:16:04.122[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:16:04.226[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:16:04.226[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:16:04.331[0m Dec 26 10:16:04.331: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-9qgxf" in namespace "azurefile-2790" to be "Succeeded or Failed" Dec 26 10:16:04.446: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Pending", Reason="", readiness=false. Elapsed: 114.359272ms Dec 26 10:16:06.549: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Pending", Reason="", readiness=false. Elapsed: 2.218042377s Dec 26 10:16:08.550: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Pending", Reason="", readiness=false. Elapsed: 4.218543847s Dec 26 10:16:10.579: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Running", Reason="", readiness=true. Elapsed: 6.247842983s Dec 26 10:16:12.553: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Running", Reason="", readiness=true. Elapsed: 8.222255042s Dec 26 10:16:14.549: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Running", Reason="", readiness=false. Elapsed: 10.217996726s Dec 26 10:16:16.550: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Running", Reason="", readiness=false. Elapsed: 12.218666639s Dec 26 10:16:18.549: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Running", Reason="", readiness=false. Elapsed: 14.218196197s Dec 26 10:16:20.551: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Running", Reason="", readiness=false. Elapsed: 16.219813523s Dec 26 10:16:22.550: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Running", Reason="", readiness=false. Elapsed: 18.218793346s Dec 26 10:16:24.549: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Succeeded", Reason="", readiness=false. Elapsed: 20.217886557s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:16:24.549[0m Dec 26 10:16:24.549: INFO: Pod "azurefile-volume-tester-9qgxf" satisfied condition "Succeeded or Failed" Dec 26 10:16:24.549: INFO: deleting Pod "azurefile-2790"/"azurefile-volume-tester-9qgxf" Dec 26 10:16:24.673: INFO: Pod azurefile-volume-tester-9qgxf has the following logs: hello world [1mSTEP:[0m Deleting pod azurefile-volume-tester-9qgxf in namespace azurefile-2790 [38;5;243m12/26/22 10:16:24.673[0m Dec 26 10:16:24.798: INFO: deleting PVC "azurefile-2790"/"pvc-mpfw4" Dec 26 10:16:24.798: INFO: Deleting PersistentVolumeClaim "pvc-mpfw4" ... skipping 31 lines ... Dec 26 10:16:03.912: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-mpfw4] to have phase Bound Dec 26 10:16:04.020: INFO: PersistentVolumeClaim pvc-mpfw4 found and phase=Bound (107.992033ms) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:16:04.02[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:16:04.122[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:16:04.226[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:16:04.226[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:16:04.331[0m Dec 26 10:16:04.331: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-9qgxf" in namespace "azurefile-2790" to be "Succeeded or Failed" Dec 26 10:16:04.446: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Pending", Reason="", readiness=false. Elapsed: 114.359272ms Dec 26 10:16:06.549: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Pending", Reason="", readiness=false. Elapsed: 2.218042377s Dec 26 10:16:08.550: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Pending", Reason="", readiness=false. Elapsed: 4.218543847s Dec 26 10:16:10.579: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Running", Reason="", readiness=true. Elapsed: 6.247842983s Dec 26 10:16:12.553: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Running", Reason="", readiness=true. Elapsed: 8.222255042s Dec 26 10:16:14.549: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Running", Reason="", readiness=false. Elapsed: 10.217996726s Dec 26 10:16:16.550: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Running", Reason="", readiness=false. Elapsed: 12.218666639s Dec 26 10:16:18.549: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Running", Reason="", readiness=false. Elapsed: 14.218196197s Dec 26 10:16:20.551: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Running", Reason="", readiness=false. Elapsed: 16.219813523s Dec 26 10:16:22.550: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Running", Reason="", readiness=false. Elapsed: 18.218793346s Dec 26 10:16:24.549: INFO: Pod "azurefile-volume-tester-9qgxf": Phase="Succeeded", Reason="", readiness=false. Elapsed: 20.217886557s [1mSTEP:[0m Saw pod success [38;5;243m12/26/22 10:16:24.549[0m Dec 26 10:16:24.549: INFO: Pod "azurefile-volume-tester-9qgxf" satisfied condition "Succeeded or Failed" Dec 26 10:16:24.549: INFO: deleting Pod "azurefile-2790"/"azurefile-volume-tester-9qgxf" Dec 26 10:16:24.673: INFO: Pod azurefile-volume-tester-9qgxf has the following logs: hello world [1mSTEP:[0m Deleting pod azurefile-volume-tester-9qgxf in namespace azurefile-2790 [38;5;243m12/26/22 10:16:24.673[0m Dec 26 10:16:24.798: INFO: deleting PVC "azurefile-2790"/"pvc-mpfw4" Dec 26 10:16:24.798: INFO: Deleting PersistentVolumeClaim "pvc-mpfw4" ... skipping 258 lines ... Dec 26 10:21:45.978: INFO: PersistentVolumeClaim pvc-shqzb found but phase is Pending instead of Bound. Dec 26 10:21:48.121: INFO: PersistentVolumeClaim pvc-shqzb found but phase is Pending instead of Bound. Dec 26 10:21:50.226: INFO: PersistentVolumeClaim pvc-shqzb found but phase is Pending instead of Bound. Dec 26 10:21:52.330: INFO: PersistentVolumeClaim pvc-shqzb found but phase is Pending instead of Bound. Dec 26 10:21:54.434: INFO: PersistentVolumeClaim pvc-shqzb found but phase is Pending instead of Bound. Dec 26 10:21:56.539: INFO: PersistentVolumeClaim pvc-shqzb found but phase is Pending instead of Bound. Dec 26 10:21:58.539: INFO: Unexpected error: <*errors.errorString | 0xc000d035c0>: { s: "PersistentVolumeClaims [pvc-shqzb] not all in phase Bound within 5m0s", } Dec 26 10:21:58.539: FAIL: PersistentVolumeClaims [pvc-shqzb] not all in phase Bound within 5m0s Full Stack Trace sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*TestPersistentVolumeClaim).WaitForBound(_) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:221 +0x19d sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*VolumeDetails).SetupDynamicPersistentVolumeClaim(0xc000b1bc70, {0x27d5520?, 0xc000531860}, 0xc000a8d1e0, {0x7f38e00b39d8, 0xc00076a1c0}, 0x0?) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/specs.go:238 +0x719 ... skipping 3 lines ... /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/dynamically_provisioned_cmd_volume_tester.go:41 +0x111 sigs.k8s.io/azurefile-csi-driver/test/e2e.glob..func1.3() /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:150 +0x5f5 [1mSTEP:[0m dump namespace information after failure [38;5;243m12/26/22 10:21:58.54[0m [1mSTEP:[0m Destroying namespace "azurefile-5194" for this suite. [38;5;243m12/26/22 10:21:58.54[0m [38;5;243m------------------------------[0m [38;5;9m• [FAILED] [302.686 seconds][0m Dynamic Provisioning [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:43[0m [38;5;9m[1m[It] should create a volume on demand with mount options [kubernetes.io/azure-file] [file.csi.azure.com] [Windows][0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:107[0m [38;5;243mBegin Captured GinkgoWriter Output >>[0m ... skipping 149 lines ... Dec 26 10:21:45.978: INFO: PersistentVolumeClaim pvc-shqzb found but phase is Pending instead of Bound. Dec 26 10:21:48.121: INFO: PersistentVolumeClaim pvc-shqzb found but phase is Pending instead of Bound. Dec 26 10:21:50.226: INFO: PersistentVolumeClaim pvc-shqzb found but phase is Pending instead of Bound. Dec 26 10:21:52.330: INFO: PersistentVolumeClaim pvc-shqzb found but phase is Pending instead of Bound. Dec 26 10:21:54.434: INFO: PersistentVolumeClaim pvc-shqzb found but phase is Pending instead of Bound. Dec 26 10:21:56.539: INFO: PersistentVolumeClaim pvc-shqzb found but phase is Pending instead of Bound. Dec 26 10:21:58.539: INFO: Unexpected error: <*errors.errorString | 0xc000d035c0>: { s: "PersistentVolumeClaims [pvc-shqzb] not all in phase Bound within 5m0s", } Dec 26 10:21:58.539: FAIL: PersistentVolumeClaims [pvc-shqzb] not all in phase Bound within 5m0s Full Stack Trace sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*TestPersistentVolumeClaim).WaitForBound(_) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:221 +0x19d sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*VolumeDetails).SetupDynamicPersistentVolumeClaim(0xc000b1bc70, {0x27d5520?, 0xc000531860}, 0xc000a8d1e0, {0x7f38e00b39d8, 0xc00076a1c0}, 0x0?) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/specs.go:238 +0x719 ... skipping 12 lines ... [1mThere were additional failures detected after the initial failure:[0m [38;5;13m[PANICKED][0m [38;5;13mTest Panicked[0m [38;5;13mIn [1m[DeferCleanup (Each)][0m[38;5;13m at: [1m/usr/local/go/src/runtime/panic.go:260[0m [38;5;13mruntime error: invalid memory address or nil pointer dereference[0m [38;5;13mFull Stack Trace[0m k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo.func1() /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:274 +0x5c k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo(0xc0004ce1e0) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:271 +0x179 ... skipping 24 lines ... Dec 26 10:22:00.344: INFO: PersistentVolumeClaim pvc-4g7z7 found but phase is Pending instead of Bound. Dec 26 10:22:02.447: INFO: PersistentVolumeClaim pvc-4g7z7 found and phase=Bound (2.2058732s) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:22:02.447[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:22:02.55[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:22:02.653[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:22:02.653[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:22:02.759[0m Dec 26 10:22:02.759: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-sl7xb" in namespace "azurefile-1353" to be "Succeeded or Failed" Dec 26 10:22:02.862: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 102.945455ms Dec 26 10:22:04.967: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 2.207856202s Dec 26 10:22:06.966: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 4.2070459s Dec 26 10:22:08.967: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 6.207761934s Dec 26 10:22:10.968: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 8.208192992s Dec 26 10:22:12.966: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 10.206641467s ... skipping 440 lines ... Dec 26 10:36:54.967: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.208089452s Dec 26 10:36:56.967: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.20789312s Dec 26 10:36:58.978: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.218920695s Dec 26 10:37:00.969: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.210041165s Dec 26 10:37:02.968: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.208810476s Dec 26 10:37:03.071: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.311665645s Dec 26 10:37:03.072: INFO: Unexpected error: <*pod.timeoutError | 0xc000632de0>: { msg: "timed out while waiting for pod azurefile-1353/azurefile-volume-tester-sl7xb to be Succeeded or Failed", observedObjects: [ <*v1.Pod | 0xc0008cad80>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "azurefile-volume-tester-sl7xb", GenerateName: "azurefile-volume-tester-", ... skipping 139 lines ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Dec 26 10:37:03.072: FAIL: timed out while waiting for pod azurefile-1353/azurefile-volume-tester-sl7xb to be Succeeded or Failed Full Stack Trace sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*TestPod).WaitForSuccess(0x23024db?) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:688 +0x5d sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedVolumeSubpathTester).Run(0xc00082bf08, {0x27d5520, 0xc0001fb860}, 0xf?) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/dynamically_provisioned_volume_subpath_tester.go:48 +0x1af sigs.k8s.io/azurefile-csi-driver/test/e2e.glob..func1.4() /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:185 +0x3d5 Dec 26 10:37:03.073: INFO: deleting Pod "azurefile-1353"/"azurefile-volume-tester-sl7xb" Dec 26 10:37:03.207: INFO: Error getting logs for pod azurefile-volume-tester-sl7xb: the server rejected our request for an unknown reason (get pods azurefile-volume-tester-sl7xb) [1mSTEP:[0m Deleting pod azurefile-volume-tester-sl7xb in namespace azurefile-1353 [38;5;243m12/26/22 10:37:03.207[0m Dec 26 10:37:03.311: INFO: deleting PVC "azurefile-1353"/"pvc-4g7z7" Dec 26 10:37:03.311: INFO: Deleting PersistentVolumeClaim "pvc-4g7z7" [1mSTEP:[0m waiting for claim's PV "pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9" to be deleted [38;5;243m12/26/22 10:37:03.626[0m Dec 26 10:37:03.626: INFO: Waiting up to 10m0s for PersistentVolume pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 to get deleted Dec 26 10:37:03.728: INFO: PersistentVolume pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 found and phase=Bound (102.465503ms) ... skipping 9 lines ... Dec 26 10:37:49.695: INFO: Waiting up to 5m0s for PersistentVolumeClaim azurefile-1353 to be removed Dec 26 10:37:49.798: INFO: Claim "azurefile-1353" in namespace "pvc-4g7z7" doesn't exist in the system Dec 26 10:37:49.798: INFO: deleting StorageClass azurefile-1353-file.csi.azure.com-dynamic-sc-9pkrn [1mSTEP:[0m dump namespace information after failure [38;5;243m12/26/22 10:37:49.903[0m [1mSTEP:[0m Destroying namespace "azurefile-1353" for this suite. [38;5;243m12/26/22 10:37:49.903[0m [38;5;243m------------------------------[0m [38;5;9m• [FAILED] [951.362 seconds][0m Dynamic Provisioning [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:43[0m [38;5;9m[1m[It] should create a pod with volume mount subpath [file.csi.azure.com] [Windows][0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:153[0m [38;5;243mBegin Captured GinkgoWriter Output >>[0m ... skipping 12 lines ... Dec 26 10:22:00.344: INFO: PersistentVolumeClaim pvc-4g7z7 found but phase is Pending instead of Bound. Dec 26 10:22:02.447: INFO: PersistentVolumeClaim pvc-4g7z7 found and phase=Bound (2.2058732s) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:22:02.447[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:22:02.55[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:22:02.653[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:22:02.653[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:22:02.759[0m Dec 26 10:22:02.759: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-sl7xb" in namespace "azurefile-1353" to be "Succeeded or Failed" Dec 26 10:22:02.862: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 102.945455ms Dec 26 10:22:04.967: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 2.207856202s Dec 26 10:22:06.966: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 4.2070459s Dec 26 10:22:08.967: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 6.207761934s Dec 26 10:22:10.968: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 8.208192992s Dec 26 10:22:12.966: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 10.206641467s ... skipping 440 lines ... Dec 26 10:36:54.967: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.208089452s Dec 26 10:36:56.967: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.20789312s Dec 26 10:36:58.978: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.218920695s Dec 26 10:37:00.969: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.210041165s Dec 26 10:37:02.968: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.208810476s Dec 26 10:37:03.071: INFO: Pod "azurefile-volume-tester-sl7xb": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.311665645s Dec 26 10:37:03.072: INFO: Unexpected error: <*pod.timeoutError | 0xc000632de0>: { msg: "timed out while waiting for pod azurefile-1353/azurefile-volume-tester-sl7xb to be Succeeded or Failed", observedObjects: [ <*v1.Pod | 0xc0008cad80>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "azurefile-volume-tester-sl7xb", GenerateName: "azurefile-volume-tester-", ... skipping 139 lines ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Dec 26 10:37:03.072: FAIL: timed out while waiting for pod azurefile-1353/azurefile-volume-tester-sl7xb to be Succeeded or Failed Full Stack Trace sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*TestPod).WaitForSuccess(0x23024db?) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:688 +0x5d sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedVolumeSubpathTester).Run(0xc00082bf08, {0x27d5520, 0xc0001fb860}, 0xf?) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/dynamically_provisioned_volume_subpath_tester.go:48 +0x1af sigs.k8s.io/azurefile-csi-driver/test/e2e.glob..func1.4() /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:185 +0x3d5 Dec 26 10:37:03.073: INFO: deleting Pod "azurefile-1353"/"azurefile-volume-tester-sl7xb" Dec 26 10:37:03.207: INFO: Error getting logs for pod azurefile-volume-tester-sl7xb: the server rejected our request for an unknown reason (get pods azurefile-volume-tester-sl7xb) [1mSTEP:[0m Deleting pod azurefile-volume-tester-sl7xb in namespace azurefile-1353 [38;5;243m12/26/22 10:37:03.207[0m Dec 26 10:37:03.311: INFO: deleting PVC "azurefile-1353"/"pvc-4g7z7" Dec 26 10:37:03.311: INFO: Deleting PersistentVolumeClaim "pvc-4g7z7" [1mSTEP:[0m waiting for claim's PV "pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9" to be deleted [38;5;243m12/26/22 10:37:03.626[0m Dec 26 10:37:03.626: INFO: Waiting up to 10m0s for PersistentVolume pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 to get deleted Dec 26 10:37:03.728: INFO: PersistentVolume pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 found and phase=Bound (102.465503ms) ... skipping 10 lines ... Dec 26 10:37:49.798: INFO: Claim "azurefile-1353" in namespace "pvc-4g7z7" doesn't exist in the system Dec 26 10:37:49.798: INFO: deleting StorageClass azurefile-1353-file.csi.azure.com-dynamic-sc-9pkrn [1mSTEP:[0m dump namespace information after failure [38;5;243m12/26/22 10:37:49.903[0m [1mSTEP:[0m Destroying namespace "azurefile-1353" for this suite. [38;5;243m12/26/22 10:37:49.903[0m [38;5;243m<< End Captured GinkgoWriter Output[0m [38;5;9mDec 26 10:37:03.072: timed out while waiting for pod azurefile-1353/azurefile-volume-tester-sl7xb to be Succeeded or Failed[0m [38;5;9mIn [1m[It][0m[38;5;9m at: [1m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:688[0m [1mThere were additional failures detected after the initial failure:[0m [38;5;13m[PANICKED][0m [38;5;13mTest Panicked[0m [38;5;13mIn [1m[DeferCleanup (Each)][0m[38;5;13m at: [1m/usr/local/go/src/runtime/panic.go:260[0m [38;5;13mruntime error: invalid memory address or nil pointer dereference[0m [38;5;13mFull Stack Trace[0m k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo.func1() /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:274 +0x5c k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo(0xc0004ce1e0) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:271 +0x179 ... skipping 178 lines ... Dec 26 10:42:46.336: INFO: Pod "azurefile-volume-tester-q25k6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.206745477s Dec 26 10:42:48.338: INFO: Pod "azurefile-volume-tester-q25k6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.208892625s Dec 26 10:42:50.337: INFO: Pod "azurefile-volume-tester-q25k6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.207940127s Dec 26 10:42:52.337: INFO: Pod "azurefile-volume-tester-q25k6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.207554402s Dec 26 10:42:54.346: INFO: Pod "azurefile-volume-tester-q25k6": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.216452912s Dec 26 10:42:54.449: INFO: Pod "azurefile-volume-tester-q25k6": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.320054954s Dec 26 10:42:54.451: INFO: Unexpected error: <*pod.timeoutError | 0xc000cc9aa0>: { msg: "timed out while waiting for pod azurefile-2888/azurefile-volume-tester-q25k6 to be running", observedObjects: [ <*v1.Pod | 0xc00057a000>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { ... skipping 142 lines ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Dec 26 10:42:54.451: FAIL: timed out while waiting for pod azurefile-2888/azurefile-volume-tester-q25k6 to be running Full Stack Trace sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*TestPod).WaitForRunning(0x22c9eaa?) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:693 +0x8d sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedCollocatedPodTest).Run(0xc00082bef8, {0x27d5520, 0xc000b3d1e0}, 0x9?) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/dynamically_provisioned_collocated_pod_tester.go:54 +0x296 sigs.k8s.io/azurefile-csi-driver/test/e2e.glob..func1.5() /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:237 +0x5ed Dec 26 10:42:54.451: INFO: deleting Pod "azurefile-2888"/"azurefile-volume-tester-q25k6" Dec 26 10:42:54.589: INFO: Error getting logs for pod azurefile-volume-tester-q25k6: the server rejected our request for an unknown reason (get pods azurefile-volume-tester-q25k6) [1mSTEP:[0m Deleting pod azurefile-volume-tester-q25k6 in namespace azurefile-2888 [38;5;243m12/26/22 10:42:54.589[0m Dec 26 10:42:54.694: INFO: deleting PVC "azurefile-2888"/"pvc-69pt7" Dec 26 10:42:54.695: INFO: Deleting PersistentVolumeClaim "pvc-69pt7" [1mSTEP:[0m waiting for claim's PV "pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9" to be deleted [38;5;243m12/26/22 10:42:55.008[0m Dec 26 10:42:55.008: INFO: Waiting up to 10m0s for PersistentVolume pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9 to get deleted Dec 26 10:42:55.111: INFO: PersistentVolume pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9 found and phase=Bound (102.41263ms) ... skipping 20 lines ... Dec 26 10:44:37.224: INFO: Waiting up to 5m0s for PersistentVolumeClaim azurefile-2888 to be removed Dec 26 10:44:37.330: INFO: Claim "azurefile-2888" in namespace "pvc-69pt7" doesn't exist in the system Dec 26 10:44:37.330: INFO: deleting StorageClass azurefile-2888-file.csi.azure.com-dynamic-sc-l25sm [1mSTEP:[0m dump namespace information after failure [38;5;243m12/26/22 10:44:37.451[0m [1mSTEP:[0m Destroying namespace "azurefile-2888" for this suite. [38;5;243m12/26/22 10:44:37.451[0m [38;5;243m------------------------------[0m [38;5;9m• [FAILED] [407.547 seconds][0m Dynamic Provisioning [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:43[0m [38;5;9m[1m[It] should create multiple PV objects, bind to PVCs and attach all to different pods on the same node [kubernetes.io/azure-file] [file.csi.azure.com] [Windows][0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:188[0m [38;5;243mBegin Captured GinkgoWriter Output >>[0m ... skipping 166 lines ... Dec 26 10:42:46.336: INFO: Pod "azurefile-volume-tester-q25k6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.206745477s Dec 26 10:42:48.338: INFO: Pod "azurefile-volume-tester-q25k6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.208892625s Dec 26 10:42:50.337: INFO: Pod "azurefile-volume-tester-q25k6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.207940127s Dec 26 10:42:52.337: INFO: Pod "azurefile-volume-tester-q25k6": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.207554402s Dec 26 10:42:54.346: INFO: Pod "azurefile-volume-tester-q25k6": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.216452912s Dec 26 10:42:54.449: INFO: Pod "azurefile-volume-tester-q25k6": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.320054954s Dec 26 10:42:54.451: INFO: Unexpected error: <*pod.timeoutError | 0xc000cc9aa0>: { msg: "timed out while waiting for pod azurefile-2888/azurefile-volume-tester-q25k6 to be running", observedObjects: [ <*v1.Pod | 0xc00057a000>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { ... skipping 142 lines ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Dec 26 10:42:54.451: FAIL: timed out while waiting for pod azurefile-2888/azurefile-volume-tester-q25k6 to be running Full Stack Trace sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*TestPod).WaitForRunning(0x22c9eaa?) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:693 +0x8d sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedCollocatedPodTest).Run(0xc00082bef8, {0x27d5520, 0xc000b3d1e0}, 0x9?) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/dynamically_provisioned_collocated_pod_tester.go:54 +0x296 sigs.k8s.io/azurefile-csi-driver/test/e2e.glob..func1.5() /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:237 +0x5ed Dec 26 10:42:54.451: INFO: deleting Pod "azurefile-2888"/"azurefile-volume-tester-q25k6" Dec 26 10:42:54.589: INFO: Error getting logs for pod azurefile-volume-tester-q25k6: the server rejected our request for an unknown reason (get pods azurefile-volume-tester-q25k6) [1mSTEP:[0m Deleting pod azurefile-volume-tester-q25k6 in namespace azurefile-2888 [38;5;243m12/26/22 10:42:54.589[0m Dec 26 10:42:54.694: INFO: deleting PVC "azurefile-2888"/"pvc-69pt7" Dec 26 10:42:54.695: INFO: Deleting PersistentVolumeClaim "pvc-69pt7" [1mSTEP:[0m waiting for claim's PV "pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9" to be deleted [38;5;243m12/26/22 10:42:55.008[0m Dec 26 10:42:55.008: INFO: Waiting up to 10m0s for PersistentVolume pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9 to get deleted Dec 26 10:42:55.111: INFO: PersistentVolume pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9 found and phase=Bound (102.41263ms) ... skipping 29 lines ... [1mThere were additional failures detected after the initial failure:[0m [38;5;13m[PANICKED][0m [38;5;13mTest Panicked[0m [38;5;13mIn [1m[DeferCleanup (Each)][0m[38;5;13m at: [1m/usr/local/go/src/runtime/panic.go:260[0m [38;5;13mruntime error: invalid memory address or nil pointer dereference[0m [38;5;13mFull Stack Trace[0m k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo.func1() /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:274 +0x5c k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo(0xc0004ce1e0) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:271 +0x179 ... skipping 33 lines ... Dec 26 10:44:58.886: INFO: PersistentVolumeClaim pvc-vjm89 found but phase is Pending instead of Bound. Dec 26 10:45:00.990: INFO: PersistentVolumeClaim pvc-vjm89 found and phase=Bound (21.14562336s) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:45:00.99[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:45:01.093[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:45:01.196[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:45:01.196[0m [1mSTEP:[0m checking that the pods command exits with an error [38;5;243m12/26/22 10:45:01.303[0m Dec 26 10:45:01.303: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-7q6s7" in namespace "azurefile-156" to be "Error status code" Dec 26 10:45:01.406: INFO: Pod "azurefile-volume-tester-7q6s7": Phase="Pending", Reason="", readiness=false. Elapsed: 102.843007ms Dec 26 10:45:03.509: INFO: Pod "azurefile-volume-tester-7q6s7": Phase="Pending", Reason="", readiness=false. Elapsed: 2.206684614s Dec 26 10:45:05.511: INFO: Pod "azurefile-volume-tester-7q6s7": Phase="Pending", Reason="", readiness=false. Elapsed: 4.208525766s Dec 26 10:45:07.511: INFO: Pod "azurefile-volume-tester-7q6s7": Phase="Running", Reason="", readiness=true. Elapsed: 6.208136719s Dec 26 10:45:09.510: INFO: Pod "azurefile-volume-tester-7q6s7": Phase="Running", Reason="", readiness=true. Elapsed: 8.207140586s Dec 26 10:45:11.509: INFO: Pod "azurefile-volume-tester-7q6s7": Phase="Running", Reason="", readiness=false. Elapsed: 10.2062747s Dec 26 10:45:13.510: INFO: Pod "azurefile-volume-tester-7q6s7": Phase="Failed", Reason="", readiness=false. Elapsed: 12.207429691s [1mSTEP:[0m Saw pod failure [38;5;243m12/26/22 10:45:13.51[0m Dec 26 10:45:13.510: INFO: Pod "azurefile-volume-tester-7q6s7" satisfied condition "Error status code" [1mSTEP:[0m checking that pod logs contain expected message [38;5;243m12/26/22 10:45:13.511[0m Dec 26 10:45:13.644: INFO: deleting Pod "azurefile-156"/"azurefile-volume-tester-7q6s7" Dec 26 10:45:13.752: INFO: Pod azurefile-volume-tester-7q6s7 has the following logs: out-file : Access to the path 'C:\mnt\test-1\data' is denied. At line:1 char:1 + echo $null >> C:\mnt\test-1\data + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ... skipping 45 lines ... Dec 26 10:44:58.886: INFO: PersistentVolumeClaim pvc-vjm89 found but phase is Pending instead of Bound. Dec 26 10:45:00.990: INFO: PersistentVolumeClaim pvc-vjm89 found and phase=Bound (21.14562336s) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:45:00.99[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:45:01.093[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:45:01.196[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:45:01.196[0m [1mSTEP:[0m checking that the pods command exits with an error [38;5;243m12/26/22 10:45:01.303[0m Dec 26 10:45:01.303: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-7q6s7" in namespace "azurefile-156" to be "Error status code" Dec 26 10:45:01.406: INFO: Pod "azurefile-volume-tester-7q6s7": Phase="Pending", Reason="", readiness=false. Elapsed: 102.843007ms Dec 26 10:45:03.509: INFO: Pod "azurefile-volume-tester-7q6s7": Phase="Pending", Reason="", readiness=false. Elapsed: 2.206684614s Dec 26 10:45:05.511: INFO: Pod "azurefile-volume-tester-7q6s7": Phase="Pending", Reason="", readiness=false. Elapsed: 4.208525766s Dec 26 10:45:07.511: INFO: Pod "azurefile-volume-tester-7q6s7": Phase="Running", Reason="", readiness=true. Elapsed: 6.208136719s Dec 26 10:45:09.510: INFO: Pod "azurefile-volume-tester-7q6s7": Phase="Running", Reason="", readiness=true. Elapsed: 8.207140586s Dec 26 10:45:11.509: INFO: Pod "azurefile-volume-tester-7q6s7": Phase="Running", Reason="", readiness=false. Elapsed: 10.2062747s Dec 26 10:45:13.510: INFO: Pod "azurefile-volume-tester-7q6s7": Phase="Failed", Reason="", readiness=false. Elapsed: 12.207429691s [1mSTEP:[0m Saw pod failure [38;5;243m12/26/22 10:45:13.51[0m Dec 26 10:45:13.510: INFO: Pod "azurefile-volume-tester-7q6s7" satisfied condition "Error status code" [1mSTEP:[0m checking that pod logs contain expected message [38;5;243m12/26/22 10:45:13.511[0m Dec 26 10:45:13.644: INFO: deleting Pod "azurefile-156"/"azurefile-volume-tester-7q6s7" Dec 26 10:45:13.752: INFO: Pod azurefile-volume-tester-7q6s7 has the following logs: out-file : Access to the path 'C:\mnt\test-1\data' is denied. At line:1 char:1 + echo $null >> C:\mnt\test-1\data + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ... skipping 326 lines ... Dec 26 10:46:24.949: INFO: PersistentVolumeClaim pvc-xl5mf found but phase is Pending instead of Bound. Dec 26 10:46:27.052: INFO: PersistentVolumeClaim pvc-xl5mf found and phase=Bound (2.206245907s) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:46:27.052[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:46:27.155[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:46:27.259[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:46:27.259[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:46:27.365[0m Dec 26 10:46:27.365: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-qvrbv" in namespace "azurefile-2546" to be "Succeeded or Failed" Dec 26 10:46:27.468: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 103.051142ms Dec 26 10:46:29.572: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 2.206888908s Dec 26 10:46:31.574: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 4.208603604s Dec 26 10:46:33.573: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 6.207809888s Dec 26 10:46:35.572: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 8.206587986s Dec 26 10:46:37.572: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 10.206600874s ... skipping 440 lines ... Dec 26 11:01:19.574: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.208626761s Dec 26 11:01:21.574: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.208825933s Dec 26 11:01:23.573: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.207648073s Dec 26 11:01:25.573: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.207986s Dec 26 11:01:27.574: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.209302806s Dec 26 11:01:27.678: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.312776637s Dec 26 11:01:27.679: INFO: Unexpected error: <*pod.timeoutError | 0xc0009d16b0>: { msg: "timed out while waiting for pod azurefile-2546/azurefile-volume-tester-qvrbv to be Succeeded or Failed", observedObjects: [ <*v1.Pod | 0xc000b9e000>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "azurefile-volume-tester-qvrbv", GenerateName: "azurefile-volume-tester-", ... skipping 139 lines ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Dec 26 11:01:27.679: FAIL: timed out while waiting for pod azurefile-2546/azurefile-volume-tester-qvrbv to be Succeeded or Failed Full Stack Trace sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*TestPod).WaitForSuccess(0x23024db?) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:688 +0x5d sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedResizeVolumeTest).Run(0xc00082bf08, {0x27d5520, 0xc000fc04e0}, 0xc0008a74a0) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/dynamically_provisioned_resize_volume_tester.go:73 +0x1da sigs.k8s.io/azurefile-csi-driver/test/e2e.glob..func1.10() /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:390 +0x2f5 Dec 26 11:01:27.679: INFO: deleting Pod "azurefile-2546"/"azurefile-volume-tester-qvrbv" Dec 26 11:01:27.819: INFO: Error getting logs for pod azurefile-volume-tester-qvrbv: the server rejected our request for an unknown reason (get pods azurefile-volume-tester-qvrbv) [1mSTEP:[0m Deleting pod azurefile-volume-tester-qvrbv in namespace azurefile-2546 [38;5;243m12/26/22 11:01:27.819[0m Dec 26 11:01:27.927: INFO: deleting PVC "azurefile-2546"/"pvc-xl5mf" Dec 26 11:01:27.927: INFO: Deleting PersistentVolumeClaim "pvc-xl5mf" [1mSTEP:[0m waiting for claim's PV "pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7" to be deleted [38;5;243m12/26/22 11:01:28.24[0m Dec 26 11:01:28.240: INFO: Waiting up to 10m0s for PersistentVolume pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 to get deleted Dec 26 11:01:28.343: INFO: PersistentVolume pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 found and phase=Bound (102.781955ms) ... skipping 26 lines ... Dec 26 10:46:24.949: INFO: PersistentVolumeClaim pvc-xl5mf found but phase is Pending instead of Bound. Dec 26 10:46:27.052: INFO: PersistentVolumeClaim pvc-xl5mf found and phase=Bound (2.206245907s) [1mSTEP:[0m checking the PVC [38;5;243m12/26/22 10:46:27.052[0m [1mSTEP:[0m validating provisioned PV [38;5;243m12/26/22 10:46:27.155[0m [1mSTEP:[0m checking the PV [38;5;243m12/26/22 10:46:27.259[0m [1mSTEP:[0m deploying the pod [38;5;243m12/26/22 10:46:27.259[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m12/26/22 10:46:27.365[0m Dec 26 10:46:27.365: INFO: Waiting up to 15m0s for pod "azurefile-volume-tester-qvrbv" in namespace "azurefile-2546" to be "Succeeded or Failed" Dec 26 10:46:27.468: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 103.051142ms Dec 26 10:46:29.572: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 2.206888908s Dec 26 10:46:31.574: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 4.208603604s Dec 26 10:46:33.573: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 6.207809888s Dec 26 10:46:35.572: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 8.206587986s Dec 26 10:46:37.572: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 10.206600874s ... skipping 440 lines ... Dec 26 11:01:19.574: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.208626761s Dec 26 11:01:21.574: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.208825933s Dec 26 11:01:23.573: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.207648073s Dec 26 11:01:25.573: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.207986s Dec 26 11:01:27.574: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.209302806s Dec 26 11:01:27.678: INFO: Pod "azurefile-volume-tester-qvrbv": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.312776637s Dec 26 11:01:27.679: INFO: Unexpected error: <*pod.timeoutError | 0xc0009d16b0>: { msg: "timed out while waiting for pod azurefile-2546/azurefile-volume-tester-qvrbv to be Succeeded or Failed", observedObjects: [ <*v1.Pod | 0xc000b9e000>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "azurefile-volume-tester-qvrbv", GenerateName: "azurefile-volume-tester-", ... skipping 139 lines ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Dec 26 11:01:27.679: FAIL: timed out while waiting for pod azurefile-2546/azurefile-volume-tester-qvrbv to be Succeeded or Failed Full Stack Trace sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*TestPod).WaitForSuccess(0x23024db?) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:688 +0x5d sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedResizeVolumeTest).Run(0xc00082bf08, {0x27d5520, 0xc000fc04e0}, 0xc0008a74a0) /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/dynamically_provisioned_resize_volume_tester.go:73 +0x1da sigs.k8s.io/azurefile-csi-driver/test/e2e.glob..func1.10() /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:390 +0x2f5 Dec 26 11:01:27.679: INFO: deleting Pod "azurefile-2546"/"azurefile-volume-tester-qvrbv" Dec 26 11:01:27.819: INFO: Error getting logs for pod azurefile-volume-tester-qvrbv: the server rejected our request for an unknown reason (get pods azurefile-volume-tester-qvrbv) [1mSTEP:[0m Deleting pod azurefile-volume-tester-qvrbv in namespace azurefile-2546 [38;5;243m12/26/22 11:01:27.819[0m Dec 26 11:01:27.927: INFO: deleting PVC "azurefile-2546"/"pvc-xl5mf" Dec 26 11:01:27.927: INFO: Deleting PersistentVolumeClaim "pvc-xl5mf" [1mSTEP:[0m waiting for claim's PV "pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7" to be deleted [38;5;243m12/26/22 11:01:28.24[0m Dec 26 11:01:28.240: INFO: Waiting up to 10m0s for PersistentVolume pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 to get deleted Dec 26 11:01:28.343: INFO: PersistentVolume pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 found and phase=Bound (102.781955ms) ... skipping 28 lines ... | ginkgo.By(fmt.Sprintf("waiting for claim's PV %q to be deleted", t.persistentVolume.Name)) [1m[38;5;214m> err := e2epv.WaitForPersistentVolumeDeleted(t.client, t.persistentVolume.Name, 5*time.Second, 10*time.Minute)[0m | framework.ExpectNoError(err) | } [38;5;243mpanic({0x20eb580, 0xc0004b48c0})[0m [38;5;243m/usr/local/go/src/runtime/panic.go:890[0m [38;5;243mgithub.com/onsi/ginkgo/v2.Fail({0xc000cecb80, 0x7b}, {0xc00082b6f0?, 0x228f280?, 0xc00082b710?})[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/github.com/onsi/ginkgo/v2/core_dsl.go:352[0m [38;5;243mk8s.io/kubernetes/test/e2e/framework.Fail({0xc0007e2000, 0x66}, {0xc00082b788?, 0xc0007e2000?, 0xc00082b7b0?})[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/log.go:61[0m [38;5;243mk8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0x1, {0x27a1320, 0xc0009d16b0}, {0x0?, 0xc001069d00?, 0xe?})[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/expect.go:76[0m [38;5;243mk8s.io/kubernetes/test/e2e/framework.ExpectNoError(...)[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/expect.go:43[0m [38;5;214m[1m> sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*TestPod).WaitForSuccess(0x23024db?)[0m ... skipping 3 lines ... [1m[38;5;214m> framework.ExpectNoError(err)[0m | } | [38;5;214m[1m> sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedResizeVolumeTest).Run(0xc00082bf08, {0x27d5520, 0xc000fc04e0}, 0xc0008a74a0)[0m [38;5;214m[1m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/dynamically_provisioned_resize_volume_tester.go:73[0m | defer tpod.Cleanup() | ginkgo.By("checking that the pods command exits with no error") [1m[38;5;214m> tpod.WaitForSuccess()[0m | | pvcName := tpod.pod.Spec.Volumes[0].VolumeSource.PersistentVolumeClaim.ClaimName [38;5;214m[1m> sigs.k8s.io/azurefile-csi-driver/test/e2e.glob..func1.10()[0m [38;5;214m[1m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:390[0m | StorageClassParameters: scParameters, ... skipping 31 lines ... | azurefileDriver = azurefile.NewDriver(&driverOptions) [1m[38;5;214m> go func() {[0m | os.Setenv("AZURE_CREDENTIAL_FILE", credentials.TempAzureCredentialFilePath) | azurefileDriver.Run(fmt.Sprintf("unix:///tmp/csi-%s.sock", uuid.NewUUID().String()), kubeconfig, false) [1mThere were additional failures detected after the initial failure:[0m [38;5;9m[FAILED][0m [38;5;9mDec 26 11:01:27.679: timed out while waiting for pod azurefile-2546/azurefile-volume-tester-qvrbv to be Succeeded or Failed[0m [38;5;9mIn [1m[DeferCleanup (Each)][0m[38;5;9m at: [1m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:688[0m [38;5;243m------------------------------[0m [38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m[38;5;14mS[0m [38;5;243m------------------------------[0m [1m[AfterSuite] [0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/suite_test.go:148[0m ... skipping 62 lines ... Streaming logs below: I1226 10:12:25.973130 1 azurefile.go:299] driver userAgent: file.csi.azure.com/e2e-e60d471c240439ae2631573c35838e7a7cb19931 e2e-test I1226 10:12:25.973284 1 azure.go:65] set QPS(25.000000) and QPS Burst(50) for driver kubeClient I1226 10:12:25.973654 1 azure.go:80] reading cloud config from secret kube-system/azure-cloud-provider I1226 10:12:26.002248 1 round_trippers.go:553] GET https://10.0.0.1:443/api/v1/namespaces/kube-system/secrets/azure-cloud-provider 404 Not Found in 28 milliseconds I1226 10:12:26.003849 1 azure.go:87] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I1226 10:12:26.003926 1 azure.go:92] could not read cloud config from secret kube-system/azure-cloud-provider I1226 10:12:26.003982 1 azure.go:102] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I1226 10:12:26.004085 1 azure.go:110] read cloud config from file: /etc/kubernetes/azure.json successfully I1226 10:12:26.005326 1 azure_auth.go:245] Using AzurePublicCloud environment I1226 10:12:26.005439 1 azure_auth.go:130] azure: using client_id+client_secret to retrieve access token I1226 10:12:26.005623 1 azure.go:775] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000 ... skipping 30 lines ... I1226 10:12:26.005942 1 azure_vmasclient.go:70] Azure AvailabilitySetsClient (read ops) using rate limit config: QPS=100, bucket=1000 I1226 10:12:26.005947 1 azure_vmasclient.go:73] Azure AvailabilitySetsClient (write ops) using rate limit config: QPS=100, bucket=1000 I1226 10:12:26.005981 1 azure.go:1006] attach/detach disk operation rate limit QPS: 6.000000, Bucket: 10 I1226 10:12:26.005990 1 azure.go:140] disable UseInstanceMetadata for controller server I1226 10:12:26.005996 1 azure.go:143] starting controller server... I1226 10:12:26.006008 1 azurefile.go:304] cloud: AzurePublicCloud, location: uksouth, rg: kubetest-vn9xafm4, VnetName: k8s-vnet-69397122, VnetResourceGroup: , SubnetName: k8s-subnet I1226 10:12:26.008859 1 mount_linux.go:287] 'umount /tmp/kubelet-detect-safe-umount2702421469' failed with: exit status 32, output: umount: /tmp/kubelet-detect-safe-umount2702421469: must be superuser to unmount. I1226 10:12:26.008887 1 mount_linux.go:289] Detected umount with unsafe 'not mounted' behavior I1226 10:12:26.008979 1 driver.go:93] Enabling controller service capability: CREATE_DELETE_VOLUME I1226 10:12:26.009028 1 driver.go:93] Enabling controller service capability: PUBLISH_UNPUBLISH_VOLUME I1226 10:12:26.009063 1 driver.go:93] Enabling controller service capability: CREATE_DELETE_SNAPSHOT I1226 10:12:26.009104 1 driver.go:93] Enabling controller service capability: EXPAND_VOLUME I1226 10:12:26.009118 1 driver.go:93] Enabling controller service capability: SINGLE_NODE_MULTI_WRITER ... skipping 518 lines ... X-Content-Type-Options: nosniff Cache-Control: no-cache Pragma: no-cache X-Ms-Failure-Cause: gateway Strict-Transport-Security: max-age=31536000; includeSubDomains I1226 10:16:57.675259 1 util.go:124] Send.sendRequest got response with ContentLength 256, StatusCode 404 and responseBody length 256 I1226 10:16:57.675520 1 azure_privatednsclient.go:215] Received error in privatednszone.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:16:57.675974 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.029733589 request="private_dns_zones_get" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="ResourceNotFound" I1226 10:16:57.676152 1 azure_storageaccount.go:232] get private dns zone privatelink.file.core.windows.net returned with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:16:57.676308 1 azure_storageaccount.go:444] Creating private dns zone(privatelink.file.core.windows.net) in resourceGroup (kubetest-vn9xafm4) (2022-12-26T10:16:57.6765694Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net?api-version=2018-09-01 Content-Type: application/json; charset=utf-8 User-Agent: Go/go1.19.4 (amd64-linux) go-autorest/v14.2.1 file.csi.azure.com/e2e-e60d471c240439ae2631573c35838e7a7cb19931 e2e-test Authorization: **REDACTED** (2022-12-26T10:16:57.6767712Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net?api-version=2018-09-01 ... skipping 140 lines ... X-Ms-Correlation-Request-Id: 9e5c6275-8077-494b-aaba-c746658bcb9e Strict-Transport-Security: max-age=31536000; includeSubDomains X-Content-Type-Options: nosniff Date: Mon, 26 Dec 2022 10:17:30 GMT Content-Length: 309 I1226 10:17:31.021964 1 util.go:124] Send.sendRequest got response with ContentLength 309, StatusCode 404 and responseBody length 309 I1226 10:17:31.021990 1 azure_virtualnetworklinksclient.go:212] Received error in virtualnetworklink.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fd425791b963449daa69928-vnetlink, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fd425791b963449daa69928-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:17:31.022146 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.044794487 request="virtual_network_links_get" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="ResourceNotFound" I1226 10:17:31.022208 1 azure_storageaccount.go:242] get virtual link for vnet(k8s-vnet-69397122) and DNS Zone(privatelink.file.core.windows.net) returned with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fd425791b963449daa69928-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:17:31.022226 1 azure_storageaccount.go:458] Creating virtual link for vnet(fd425791b963449daa69928-vnetlink) and DNS Zone(privatelink.file.core.windows.net) in resourceGroup(kubetest-vn9xafm4) (2022-12-26T10:17:31.0224062Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fd425791b963449daa69928-vnetlink?api-version=2020-06-01 Content-Type: application/json; charset=utf-8 User-Agent: Go/go1.19.4 (amd64-linux) go-autorest/v14.2.1 file.csi.azure.com/e2e-e60d471c240439ae2631573c35838e7a7cb19931 e2e-test Authorization: **REDACTED** (2022-12-26T10:17:31.0224565Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fd425791b963449daa69928-vnetlink?api-version=2020-06-01 ... skipping 384 lines ... Server: Microsoft-HTTPAPI/2.0,Microsoft-HTTPAPI/2.0 Content-Length: 634 X-Ms-Request-Id: ddc1fe0d-d435-43d1-a822-db055343f3f6 Strict-Transport-Security: max-age=31536000; includeSubDomains X-Ms-Ratelimit-Remaining-Subscription-Writes: 1190 X-Content-Type-Options: nosniff I1226 10:17:56.561731 1 azure_armclient.go:301] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fd425791b963449daa69928-pvtendpoint?api-version=2021-08-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fd425791b963449daa69928-pvtendpoint/privateLinkServiceConnections/fd425791b963449daa69928-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/fd425791b963449daa69928).", "details": [] } } I1226 10:17:56.561756 1 azure_armclient.go:504] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fd425791b963449daa69928-pvtendpoint, error: %!s(<nil>) I1226 10:17:56.561771 1 azure_privateendpointclient.go:141] Received error in privateendpoint.put.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fd425791b963449daa69928-pvtendpoint, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fd425791b963449daa69928-pvtendpoint/privateLinkServiceConnections/fd425791b963449daa69928-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/fd425791b963449daa69928).", "details": [] } } I1226 10:17:56.561893 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.296937167 request="private_endpoints_create_or_update" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="OnlyOneGroupIdPermitted" E1226 10:17:56.561954 1 utils.go:81] GRPC error: rpc error: code = Internal desc = failed to ensure storage account: create private endpoint for storage account(fd425791b963449daa69928), resourceGroup(kubetest-vn9xafm4): Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fd425791b963449daa69928-pvtendpoint/privateLinkServiceConnections/fd425791b963449daa69928-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/fd425791b963449daa69928).", "details": [] } } I1226 10:17:57.564281 1 utils.go:76] GRPC call: /csi.v1.Controller/CreateVolume ... skipping 102 lines ... X-Content-Type-Options: nosniff Cache-Control: no-cache Pragma: no-cache Date: Mon, 26 Dec 2022 10:17:57 GMT Content-Length: 309 I1226 10:17:57.822410 1 util.go:124] Send.sendRequest got response with ContentLength 309, StatusCode 404 and responseBody length 309 I1226 10:17:57.822705 1 azure_virtualnetworklinksclient.go:212] Received error in virtualnetworklink.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fe628abe53fe444b5a2cd13-vnetlink, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fe628abe53fe444b5a2cd13-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:17:57.822800 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.027224655 request="virtual_network_links_get" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="ResourceNotFound" I1226 10:17:57.822847 1 azure_storageaccount.go:242] get virtual link for vnet(k8s-vnet-69397122) and DNS Zone(privatelink.file.core.windows.net) returned with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fe628abe53fe444b5a2cd13-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:17:57.822863 1 azure_storageaccount.go:458] Creating virtual link for vnet(fe628abe53fe444b5a2cd13-vnetlink) and DNS Zone(privatelink.file.core.windows.net) in resourceGroup(kubetest-vn9xafm4) (2022-12-26T10:17:57.8230257Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fe628abe53fe444b5a2cd13-vnetlink?api-version=2020-06-01 Content-Type: application/json; charset=utf-8 User-Agent: Go/go1.19.4 (amd64-linux) go-autorest/v14.2.1 file.csi.azure.com/e2e-e60d471c240439ae2631573c35838e7a7cb19931 e2e-test Authorization: **REDACTED** (2022-12-26T10:17:57.8231534Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fe628abe53fe444b5a2cd13-vnetlink?api-version=2020-06-01 ... skipping 345 lines ... X-Ms-Routing-Request-Id: UKSOUTH:20221226T101819Z:eb589104-068e-453a-9e63-c7e84c0e819b Pragma: no-cache Content-Length: 634 Content-Type: application/json; charset=utf-8 Expires: -1 Server: Microsoft-HTTPAPI/2.0,Microsoft-HTTPAPI/2.0 I1226 10:18:19.759313 1 azure_armclient.go:301] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fe628abe53fe444b5a2cd13-pvtendpoint?api-version=2021-08-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fe628abe53fe444b5a2cd13-pvtendpoint/privateLinkServiceConnections/fe628abe53fe444b5a2cd13-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/fe628abe53fe444b5a2cd13).", "details": [] } } I1226 10:18:19.759341 1 azure_armclient.go:504] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fe628abe53fe444b5a2cd13-pvtendpoint, error: %!s(<nil>) I1226 10:18:19.759356 1 azure_privateendpointclient.go:141] Received error in privateendpoint.put.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fe628abe53fe444b5a2cd13-pvtendpoint, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fe628abe53fe444b5a2cd13-pvtendpoint/privateLinkServiceConnections/fe628abe53fe444b5a2cd13-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/fe628abe53fe444b5a2cd13).", "details": [] } } I1226 10:18:19.759528 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.272628241 request="private_endpoints_create_or_update" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="OnlyOneGroupIdPermitted" E1226 10:18:19.759646 1 utils.go:81] GRPC error: rpc error: code = Internal desc = failed to ensure storage account: create private endpoint for storage account(fe628abe53fe444b5a2cd13), resourceGroup(kubetest-vn9xafm4): Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fe628abe53fe444b5a2cd13-pvtendpoint/privateLinkServiceConnections/fe628abe53fe444b5a2cd13-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/fe628abe53fe444b5a2cd13).", "details": [] } } I1226 10:18:21.761426 1 utils.go:76] GRPC call: /csi.v1.Controller/CreateVolume ... skipping 102 lines ... Content-Length: 309 Expires: -1 X-Ms-Request-Id: 0ef29351-727e-4b34-aba0-20886b44639d Strict-Transport-Security: max-age=31536000; includeSubDomains X-Content-Type-Options: nosniff I1226 10:18:22.070577 1 util.go:124] Send.sendRequest got response with ContentLength 309, StatusCode 404 and responseBody length 309 I1226 10:18:22.070690 1 azure_virtualnetworklinksclient.go:212] Received error in virtualnetworklink.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/faf80bae47d2d4b60aeb17b-vnetlink, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/faf80bae47d2d4b60aeb17b-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:18:22.070939 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.02976699 request="virtual_network_links_get" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="ResourceNotFound" I1226 10:18:22.071011 1 azure_storageaccount.go:242] get virtual link for vnet(k8s-vnet-69397122) and DNS Zone(privatelink.file.core.windows.net) returned with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/faf80bae47d2d4b60aeb17b-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:18:22.071046 1 azure_storageaccount.go:458] Creating virtual link for vnet(faf80bae47d2d4b60aeb17b-vnetlink) and DNS Zone(privatelink.file.core.windows.net) in resourceGroup(kubetest-vn9xafm4) (2022-12-26T10:18:22.0712167Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/faf80bae47d2d4b60aeb17b-vnetlink?api-version=2020-06-01 Content-Type: application/json; charset=utf-8 User-Agent: Go/go1.19.4 (amd64-linux) go-autorest/v14.2.1 file.csi.azure.com/e2e-e60d471c240439ae2631573c35838e7a7cb19931 e2e-test Authorization: **REDACTED** (2022-12-26T10:18:22.0712937Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/faf80bae47d2d4b60aeb17b-vnetlink?api-version=2020-06-01 ... skipping 384 lines ... Content-Length: 634 Server: Microsoft-HTTPAPI/2.0,Microsoft-HTTPAPI/2.0 Date: Mon, 26 Dec 2022 10:18:47 GMT X-Ms-Request-Id: 53c8003a-4abf-4d7e-892b-9048cefb8579 X-Ms-Correlation-Request-Id: 4ac452a9-c11b-4b02-8c71-0c06ff7ac0ee Strict-Transport-Security: max-age=31536000; includeSubDomains I1226 10:18:47.719700 1 azure_armclient.go:301] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/faf80bae47d2d4b60aeb17b-pvtendpoint?api-version=2021-08-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/faf80bae47d2d4b60aeb17b-pvtendpoint/privateLinkServiceConnections/faf80bae47d2d4b60aeb17b-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/faf80bae47d2d4b60aeb17b).", "details": [] } } I1226 10:18:47.719728 1 azure_armclient.go:504] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/faf80bae47d2d4b60aeb17b-pvtendpoint, error: %!s(<nil>) I1226 10:18:47.719746 1 azure_privateendpointclient.go:141] Received error in privateendpoint.put.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/faf80bae47d2d4b60aeb17b-pvtendpoint, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/faf80bae47d2d4b60aeb17b-pvtendpoint/privateLinkServiceConnections/faf80bae47d2d4b60aeb17b-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/faf80bae47d2d4b60aeb17b).", "details": [] } } I1226 10:18:47.719845 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.262716246 request="private_endpoints_create_or_update" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="OnlyOneGroupIdPermitted" E1226 10:18:47.719910 1 utils.go:81] GRPC error: rpc error: code = Internal desc = failed to ensure storage account: create private endpoint for storage account(faf80bae47d2d4b60aeb17b), resourceGroup(kubetest-vn9xafm4): Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/faf80bae47d2d4b60aeb17b-pvtendpoint/privateLinkServiceConnections/faf80bae47d2d4b60aeb17b-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/faf80bae47d2d4b60aeb17b).", "details": [] } } I1226 10:18:51.721638 1 utils.go:76] GRPC call: /csi.v1.Controller/CreateVolume ... skipping 102 lines ... Cache-Control: no-cache Pragma: no-cache Content-Type: application/json; charset=utf-8 X-Ms-Request-Id: 8dccccb2-ec43-455b-948e-c77f8a07384c X-Ms-Correlation-Request-Id: 8dccccb2-ec43-455b-948e-c77f8a07384c I1226 10:18:52.627463 1 util.go:124] Send.sendRequest got response with ContentLength 309, StatusCode 404 and responseBody length 309 I1226 10:18:52.627482 1 azure_virtualnetworklinksclient.go:212] Received error in virtualnetworklink.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fb08e2de2ec5a4fe4a1e812-vnetlink, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fb08e2de2ec5a4fe4a1e812-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:18:52.627596 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.027028254 request="virtual_network_links_get" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="ResourceNotFound" I1226 10:18:52.627647 1 azure_storageaccount.go:242] get virtual link for vnet(k8s-vnet-69397122) and DNS Zone(privatelink.file.core.windows.net) returned with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fb08e2de2ec5a4fe4a1e812-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:18:52.627662 1 azure_storageaccount.go:458] Creating virtual link for vnet(fb08e2de2ec5a4fe4a1e812-vnetlink) and DNS Zone(privatelink.file.core.windows.net) in resourceGroup(kubetest-vn9xafm4) (2022-12-26T10:18:52.6277715Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fb08e2de2ec5a4fe4a1e812-vnetlink?api-version=2020-06-01 Content-Type: application/json; charset=utf-8 User-Agent: Go/go1.19.4 (amd64-linux) go-autorest/v14.2.1 file.csi.azure.com/e2e-e60d471c240439ae2631573c35838e7a7cb19931 e2e-test Authorization: **REDACTED** (2022-12-26T10:18:52.6278454Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fb08e2de2ec5a4fe4a1e812-vnetlink?api-version=2020-06-01 ... skipping 347 lines ... Server: Microsoft-HTTPAPI/2.0,Microsoft-HTTPAPI/2.0 X-Ms-Routing-Request-Id: UKSOUTH:20221226T101915Z:e05217dd-ebe5-45b8-a998-6a9b508134c3 X-Content-Type-Options: nosniff Date: Mon, 26 Dec 2022 10:19:14 GMT Content-Type: application/json; charset=utf-8 X-Ms-Request-Id: eb4dc652-bf81-4d26-8d31-8974e98ba90d I1226 10:19:15.071102 1 azure_armclient.go:301] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fb08e2de2ec5a4fe4a1e812-pvtendpoint?api-version=2021-08-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fb08e2de2ec5a4fe4a1e812-pvtendpoint/privateLinkServiceConnections/fb08e2de2ec5a4fe4a1e812-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/fb08e2de2ec5a4fe4a1e812).", "details": [] } } I1226 10:19:15.071142 1 azure_armclient.go:504] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fb08e2de2ec5a4fe4a1e812-pvtendpoint, error: %!s(<nil>) I1226 10:19:15.071162 1 azure_privateendpointclient.go:141] Received error in privateendpoint.put.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fb08e2de2ec5a4fe4a1e812-pvtendpoint, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fb08e2de2ec5a4fe4a1e812-pvtendpoint/privateLinkServiceConnections/fb08e2de2ec5a4fe4a1e812-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/fb08e2de2ec5a4fe4a1e812).", "details": [] } } I1226 10:19:15.071283 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.333606572 request="private_endpoints_create_or_update" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="OnlyOneGroupIdPermitted" E1226 10:19:15.071372 1 utils.go:81] GRPC error: rpc error: code = Internal desc = failed to ensure storage account: create private endpoint for storage account(fb08e2de2ec5a4fe4a1e812), resourceGroup(kubetest-vn9xafm4): Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fb08e2de2ec5a4fe4a1e812-pvtendpoint/privateLinkServiceConnections/fb08e2de2ec5a4fe4a1e812-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/fb08e2de2ec5a4fe4a1e812).", "details": [] } } I1226 10:19:23.073812 1 utils.go:76] GRPC call: /csi.v1.Controller/CreateVolume ... skipping 102 lines ... Strict-Transport-Security: max-age=31536000; includeSubDomains Date: Mon, 26 Dec 2022 10:19:22 GMT Content-Length: 309 Content-Type: application/json; charset=utf-8 X-Ms-Request-Id: f55a4ba3-6fcf-4ac5-ab4f-1df503300ce4 I1226 10:19:23.562422 1 util.go:124] Send.sendRequest got response with ContentLength 309, StatusCode 404 and responseBody length 309 I1226 10:19:23.562487 1 azure_virtualnetworklinksclient.go:212] Received error in virtualnetworklink.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fe90209ae944c4af983d553-vnetlink, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fe90209ae944c4af983d553-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:19:23.562646 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.04615041 request="virtual_network_links_get" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="ResourceNotFound" I1226 10:19:23.562784 1 azure_storageaccount.go:242] get virtual link for vnet(k8s-vnet-69397122) and DNS Zone(privatelink.file.core.windows.net) returned with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fe90209ae944c4af983d553-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:19:23.562803 1 azure_storageaccount.go:458] Creating virtual link for vnet(fe90209ae944c4af983d553-vnetlink) and DNS Zone(privatelink.file.core.windows.net) in resourceGroup(kubetest-vn9xafm4) (2022-12-26T10:19:23.5629745Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fe90209ae944c4af983d553-vnetlink?api-version=2020-06-01 Content-Type: application/json; charset=utf-8 User-Agent: Go/go1.19.4 (amd64-linux) go-autorest/v14.2.1 file.csi.azure.com/e2e-e60d471c240439ae2631573c35838e7a7cb19931 e2e-test Authorization: **REDACTED** (2022-12-26T10:19:23.5630226Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fe90209ae944c4af983d553-vnetlink?api-version=2020-06-01 ... skipping 345 lines ... X-Ms-Correlation-Request-Id: cdf0fa13-dea0-4eb4-aa7e-5ba84dbae3de X-Ms-Arm-Service-Request-Id: e303757a-fc0a-4015-ac95-9ee7adf17443 X-Ms-Ratelimit-Remaining-Subscription-Writes: 1178 Pragma: no-cache Expires: -1 Strict-Transport-Security: max-age=31536000; includeSubDomains I1226 10:19:45.808246 1 azure_armclient.go:301] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fe90209ae944c4af983d553-pvtendpoint?api-version=2021-08-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fe90209ae944c4af983d553-pvtendpoint/privateLinkServiceConnections/fe90209ae944c4af983d553-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/fe90209ae944c4af983d553).", "details": [] } } I1226 10:19:45.808263 1 azure_armclient.go:504] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fe90209ae944c4af983d553-pvtendpoint, error: %!s(<nil>) I1226 10:19:45.808277 1 azure_privateendpointclient.go:141] Received error in privateendpoint.put.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fe90209ae944c4af983d553-pvtendpoint, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fe90209ae944c4af983d553-pvtendpoint/privateLinkServiceConnections/fe90209ae944c4af983d553-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/fe90209ae944c4af983d553).", "details": [] } } I1226 10:19:45.808391 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.291514307 request="private_endpoints_create_or_update" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="OnlyOneGroupIdPermitted" E1226 10:19:45.808453 1 utils.go:81] GRPC error: rpc error: code = Internal desc = failed to ensure storage account: create private endpoint for storage account(fe90209ae944c4af983d553), resourceGroup(kubetest-vn9xafm4): Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fe90209ae944c4af983d553-pvtendpoint/privateLinkServiceConnections/fe90209ae944c4af983d553-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/fe90209ae944c4af983d553).", "details": [] } } I1226 10:20:01.810930 1 utils.go:76] GRPC call: /csi.v1.Controller/CreateVolume ... skipping 102 lines ... X-Ms-Correlation-Request-Id: db15d1ec-d6ec-482a-a8ba-1e6c7d8f1c20 X-Content-Type-Options: nosniff Date: Mon, 26 Dec 2022 10:20:01 GMT Content-Length: 309 Content-Type: application/json; charset=utf-8 I1226 10:20:02.213665 1 util.go:124] Send.sendRequest got response with ContentLength 309, StatusCode 404 and responseBody length 309 I1226 10:20:02.213685 1 azure_virtualnetworklinksclient.go:212] Received error in virtualnetworklink.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fc084386092164dc9b5fca8-vnetlink, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fc084386092164dc9b5fca8-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:20:02.213778 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.041911247 request="virtual_network_links_get" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="ResourceNotFound" I1226 10:20:02.213795 1 azure_storageaccount.go:242] get virtual link for vnet(k8s-vnet-69397122) and DNS Zone(privatelink.file.core.windows.net) returned with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fc084386092164dc9b5fca8-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:20:02.213805 1 azure_storageaccount.go:458] Creating virtual link for vnet(fc084386092164dc9b5fca8-vnetlink) and DNS Zone(privatelink.file.core.windows.net) in resourceGroup(kubetest-vn9xafm4) (2022-12-26T10:20:02.2138990Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fc084386092164dc9b5fca8-vnetlink?api-version=2020-06-01 Content-Type: application/json; charset=utf-8 User-Agent: Go/go1.19.4 (amd64-linux) go-autorest/v14.2.1 file.csi.azure.com/e2e-e60d471c240439ae2631573c35838e7a7cb19931 e2e-test Authorization: **REDACTED** (2022-12-26T10:20:02.2139594Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/fc084386092164dc9b5fca8-vnetlink?api-version=2020-06-01 ... skipping 384 lines ... Cache-Control: no-cache X-Ms-Request-Id: a3d0cf99-6ca7-46c4-8517-06fc498b5b1b Date: Mon, 26 Dec 2022 10:20:27 GMT Content-Length: 634 X-Ms-Routing-Request-Id: UKSOUTH:20221226T102028Z:71f5ff59-3775-48da-b2f8-d7dd5e6a5aa3 X-Ms-Correlation-Request-Id: 71f5ff59-3775-48da-b2f8-d7dd5e6a5aa3 I1226 10:20:28.461679 1 azure_armclient.go:301] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fc084386092164dc9b5fca8-pvtendpoint?api-version=2021-08-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fc084386092164dc9b5fca8-pvtendpoint/privateLinkServiceConnections/fc084386092164dc9b5fca8-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/fc084386092164dc9b5fca8).", "details": [] } } I1226 10:20:28.461712 1 azure_armclient.go:504] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fc084386092164dc9b5fca8-pvtendpoint, error: %!s(<nil>) I1226 10:20:28.461748 1 azure_privateendpointclient.go:141] Received error in privateendpoint.put.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fc084386092164dc9b5fca8-pvtendpoint, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fc084386092164dc9b5fca8-pvtendpoint/privateLinkServiceConnections/fc084386092164dc9b5fca8-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/fc084386092164dc9b5fca8).", "details": [] } } I1226 10:20:28.461938 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.312310981 request="private_endpoints_create_or_update" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="OnlyOneGroupIdPermitted" E1226 10:20:28.462118 1 utils.go:81] GRPC error: rpc error: code = Internal desc = failed to ensure storage account: create private endpoint for storage account(fc084386092164dc9b5fca8), resourceGroup(kubetest-vn9xafm4): Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/fc084386092164dc9b5fca8-pvtendpoint/privateLinkServiceConnections/fc084386092164dc9b5fca8-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/fc084386092164dc9b5fca8).", "details": [] } } I1226 10:20:32.849047 1 utils.go:76] GRPC call: /csi.v1.Identity/Probe ... skipping 105 lines ... Expires: -1 X-Ms-Failure-Cause: gateway X-Ms-Correlation-Request-Id: 7374adec-50ed-4753-a353-8df0dcf6dfd4 Strict-Transport-Security: max-age=31536000; includeSubDomains X-Content-Type-Options: nosniff I1226 10:21:00.778291 1 util.go:124] Send.sendRequest got response with ContentLength 309, StatusCode 404 and responseBody length 309 I1226 10:21:00.778367 1 azure_virtualnetworklinksclient.go:212] Received error in virtualnetworklink.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/f38065232573a4f48bbfbc8-vnetlink, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/f38065232573a4f48bbfbc8-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:21:00.778553 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.024414919 request="virtual_network_links_get" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="ResourceNotFound" I1226 10:21:00.778616 1 azure_storageaccount.go:242] get virtual link for vnet(k8s-vnet-69397122) and DNS Zone(privatelink.file.core.windows.net) returned with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/f38065232573a4f48bbfbc8-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:21:00.778627 1 azure_storageaccount.go:458] Creating virtual link for vnet(f38065232573a4f48bbfbc8-vnetlink) and DNS Zone(privatelink.file.core.windows.net) in resourceGroup(kubetest-vn9xafm4) (2022-12-26T10:21:00.7787710Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/f38065232573a4f48bbfbc8-vnetlink?api-version=2020-06-01 Content-Type: application/json; charset=utf-8 User-Agent: Go/go1.19.4 (amd64-linux) go-autorest/v14.2.1 file.csi.azure.com/e2e-e60d471c240439ae2631573c35838e7a7cb19931 e2e-test Authorization: **REDACTED** (2022-12-26T10:21:00.7788388Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/f38065232573a4f48bbfbc8-vnetlink?api-version=2020-06-01 ... skipping 345 lines ... X-Ms-Request-Id: f5b5718e-bcb6-428b-8d6e-20f7efda76a5 X-Ms-Correlation-Request-Id: a8382245-939e-40d8-817f-44b23bd7d1bf X-Ms-Arm-Service-Request-Id: d35db747-e2ea-40e0-99e0-1f225335bb1a Pragma: no-cache X-Ms-Routing-Request-Id: UKSOUTH:20221226T102123Z:a8382245-939e-40d8-817f-44b23bd7d1bf X-Content-Type-Options: nosniff I1226 10:21:23.686075 1 azure_armclient.go:301] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/f38065232573a4f48bbfbc8-pvtendpoint?api-version=2021-08-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/f38065232573a4f48bbfbc8-pvtendpoint/privateLinkServiceConnections/f38065232573a4f48bbfbc8-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/f38065232573a4f48bbfbc8).", "details": [] } } I1226 10:21:23.686095 1 azure_armclient.go:504] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/f38065232573a4f48bbfbc8-pvtendpoint, error: %!s(<nil>) I1226 10:21:23.686158 1 azure_privateendpointclient.go:141] Received error in privateendpoint.put.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/f38065232573a4f48bbfbc8-pvtendpoint, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/f38065232573a4f48bbfbc8-pvtendpoint/privateLinkServiceConnections/f38065232573a4f48bbfbc8-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/f38065232573a4f48bbfbc8).", "details": [] } } I1226 10:21:23.686329 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.368869325 request="private_endpoints_create_or_update" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="OnlyOneGroupIdPermitted" E1226 10:21:23.686434 1 utils.go:81] GRPC error: rpc error: code = Internal desc = failed to ensure storage account: create private endpoint for storage account(f38065232573a4f48bbfbc8), resourceGroup(kubetest-vn9xafm4): Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/f38065232573a4f48bbfbc8-pvtendpoint/privateLinkServiceConnections/f38065232573a4f48bbfbc8-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/f38065232573a4f48bbfbc8).", "details": [] } } I1226 10:21:32.849733 1 utils.go:76] GRPC call: /csi.v1.Identity/Probe ... skipping 228 lines ... Date: Mon, 26 Dec 2022 10:22:03 GMT Cache-Control: no-cache Pragma: no-cache Expires: -1 Content-Length: 309 I1226 10:22:03.900705 1 util.go:124] Send.sendRequest got response with ContentLength 309, StatusCode 404 and responseBody length 309 I1226 10:22:03.900771 1 azure_virtualnetworklinksclient.go:212] Received error in virtualnetworklink.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/f2d382de91e544306a75352-vnetlink, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/f2d382de91e544306a75352-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:22:03.900915 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.026396845 request="virtual_network_links_get" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="ResourceNotFound" I1226 10:22:03.900976 1 azure_storageaccount.go:242] get virtual link for vnet(k8s-vnet-69397122) and DNS Zone(privatelink.file.core.windows.net) returned with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/f2d382de91e544306a75352-vnetlink' under resource group 'kubetest-vn9xafm4' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}} I1226 10:22:03.901044 1 azure_storageaccount.go:458] Creating virtual link for vnet(f2d382de91e544306a75352-vnetlink) and DNS Zone(privatelink.file.core.windows.net) in resourceGroup(kubetest-vn9xafm4) (2022-12-26T10:22:03.9011970Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/f2d382de91e544306a75352-vnetlink?api-version=2020-06-01 Content-Type: application/json; charset=utf-8 User-Agent: Go/go1.19.4 (amd64-linux) go-autorest/v14.2.1 file.csi.azure.com/e2e-e60d471c240439ae2631573c35838e7a7cb19931 e2e-test Authorization: **REDACTED** (2022-12-26T10:22:03.9012625Z) INFO: REQUEST: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/f2d382de91e544306a75352-vnetlink?api-version=2020-06-01 ... skipping 381 lines ... Cache-Control: no-cache Pragma: no-cache X-Ms-Ratelimit-Remaining-Subscription-Writes: 1169 Date: Mon, 26 Dec 2022 10:22:28 GMT Content-Length: 634 Strict-Transport-Security: max-age=31536000; includeSubDomains I1226 10:22:28.959726 1 azure_armclient.go:301] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/f2d382de91e544306a75352-pvtendpoint?api-version=2021-08-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/f2d382de91e544306a75352-pvtendpoint/privateLinkServiceConnections/f2d382de91e544306a75352-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/f2d382de91e544306a75352).", "details": [] } } I1226 10:22:28.959741 1 azure_armclient.go:504] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/f2d382de91e544306a75352-pvtendpoint, error: %!s(<nil>) I1226 10:22:28.959755 1 azure_privateendpointclient.go:141] Received error in privateendpoint.put.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/f2d382de91e544306a75352-pvtendpoint, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/f2d382de91e544306a75352-pvtendpoint/privateLinkServiceConnections/f2d382de91e544306a75352-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/f2d382de91e544306a75352).", "details": [] } } I1226 10:22:28.959910 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.255380671 request="private_endpoints_create_or_update" resource_group="kubetest-vn9xafm4" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="" error_code="OnlyOneGroupIdPermitted" E1226 10:22:28.959992 1 utils.go:81] GRPC error: rpc error: code = Internal desc = failed to ensure storage account: create private endpoint for storage account(f2d382de91e544306a75352), resourceGroup(kubetest-vn9xafm4): Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: { "error": { "code": "OnlyOneGroupIdPermitted", "message": "Private link service connection /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Network/privateEndpoints/f2d382de91e544306a75352-pvtendpoint/privateLinkServiceConnections/f2d382de91e544306a75352-pvtsvcconn has the following group Ids: file, blob. Only one group Id is permitted when connecting to a first-party resource (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vn9xafm4/providers/Microsoft.Storage/storageAccounts/f2d382de91e544306a75352).", "details": [] } } I1226 10:22:32.849533 1 utils.go:76] GRPC call: /csi.v1.Identity/Probe ... skipping 1738 lines ... Streaming logs below: I1226 10:13:07.132389 6928 azurefile.go:299] driver userAgent: file.csi.azure.com/e2e-e60d471c240439ae2631573c35838e7a7cb19931 e2e-test I1226 10:13:07.146387 6928 azure.go:65] set QPS(25.000000) and QPS Burst(50) for driver kubeClient I1226 10:13:07.151767 6928 azure.go:80] reading cloud config from secret kube-system/azure-cloud-provider I1226 10:13:07.195911 6928 round_trippers.go:553] GET https://10.0.0.1:443/api/v1/namespaces/kube-system/secrets/azure-cloud-provider 404 Not Found in 41 milliseconds I1226 10:13:07.197123 6928 azure.go:87] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I1226 10:13:07.197123 6928 azure.go:92] could not read cloud config from secret kube-system/azure-cloud-provider I1226 10:13:07.197123 6928 azure.go:102] use default AZURE_CREDENTIAL_FILE env var: C:\k\azure.json I1226 10:13:07.205197 6928 azure.go:110] read cloud config from file: C:\k\azure.json successfully I1226 10:13:07.207645 6928 azure_auth.go:245] Using AzurePublicCloud environment I1226 10:13:07.208316 6928 azure_auth.go:130] azure: using client_id+client_secret to retrieve access token I1226 10:13:07.209580 6928 azure_diskclient.go:68] Azure DisksClient using API version: 2022-03-02 ... skipping 352 lines ... I1226 10:16:06.468237 6928 utils.go:77] GRPC request: {} I1226 10:16:06.468237 6928 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:16:06.470702 6928 utils.go:76] GRPC call: /csi.v1.Node/NodePublishVolume I1226 10:16:06.470702 6928 utils.go:77] GRPC request: {"staging_target_path":"\\var\\lib\\kubelet\\plugins\\kubernetes.io\\csi\\pv\\azurefile-2790-file.csi.azure.com-preprovsioned-pv-8jlff\\globalmount","target_path":"c:\\var\\lib\\kubelet\\pods\\7e556f2b-31bf-48b3-8147-c0f0df623952\\volumes\\kubernetes.io~csi\\azurefile-2790-file.csi.azure.com-preprovsioned-pv-8jlff\\mount","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/ephemeral":"false","csi.storage.k8s.io/pod.name":"azurefile-volume-tester-9qgxf","csi.storage.k8s.io/pod.namespace":"azurefile-2790","csi.storage.k8s.io/pod.uid":"7e556f2b-31bf-48b3-8147-c0f0df623952","csi.storage.k8s.io/serviceAccount.name":"default","shareName":"pre-provisioned-provided-credentials"},"volume_id":"kubetest-vn9xafm4#f4f4b9e44678c403d89fc3a#pre-provisioned-provided-credentials##pre-provisioned-provided-credentials#azurefile-2790"} I1226 10:16:06.470702 6928 safe_mounter_windows.go:163] IsLikelyNotMountPoint: c:\var\lib\kubelet\pods\7e556f2b-31bf-48b3-8147-c0f0df623952\volumes\kubernetes.io~csi\azurefile-2790-file.csi.azure.com-preprovsioned-pv-8jlff\mount I1226 10:16:06.470702 6928 safe_mounter_windows.go:220] Exists path: c:\var\lib\kubelet\pods\7e556f2b-31bf-48b3-8147-c0f0df623952\volumes\kubernetes.io~csi\azurefile-2790-file.csi.azure.com-preprovsioned-pv-8jlff\mount W1226 10:16:18.558209 6928 nodeserver.go:529] ReadDir c:\var\lib\kubelet\pods\7e556f2b-31bf-48b3-8147-c0f0df623952\volumes\kubernetes.io~csi\azurefile-2790-file.csi.azure.com-preprovsioned-pv-8jlff\mount failed with open c:\var\lib\kubelet\pods\7e556f2b-31bf-48b3-8147-c0f0df623952\volumes\kubernetes.io~csi\azurefile-2790-file.csi.azure.com-preprovsioned-pv-8jlff\mount: Access is denied., unmount this directory I1226 10:16:18.559355 6928 safe_mounter_windows.go:147] Unmount: c:\var\lib\kubelet\pods\7e556f2b-31bf-48b3-8147-c0f0df623952\volumes\kubernetes.io~csi\azurefile-2790-file.csi.azure.com-preprovsioned-pv-8jlff\mount I1226 10:16:18.559355 6928 safe_mounter_windows.go:133] Remove directory: c:\var\lib\kubelet\pods\7e556f2b-31bf-48b3-8147-c0f0df623952\volumes\kubernetes.io~csi\azurefile-2790-file.csi.azure.com-preprovsioned-pv-8jlff\mount E1226 10:16:18.561344 6928 utils.go:81] GRPC error: rpc error: code = Internal desc = Could not mount target c:\var\lib\kubelet\pods\7e556f2b-31bf-48b3-8147-c0f0df623952\volumes\kubernetes.io~csi\azurefile-2790-file.csi.azure.com-preprovsioned-pv-8jlff\mount: open c:\var\lib\kubelet\pods\7e556f2b-31bf-48b3-8147-c0f0df623952\volumes\kubernetes.io~csi\azurefile-2790-file.csi.azure.com-preprovsioned-pv-8jlff\mount: Access is denied. I1226 10:16:33.026848 6928 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:16:33.026948 6928 utils.go:77] GRPC request: {} I1226 10:16:33.026948 6928 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:17:03.027051 6928 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:17:03.027051 6928 utils.go:77] GRPC request: {} I1226 10:17:03.027120 6928 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 138 lines ... I1226 10:37:54.334635 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:37:54.334635 6928 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:37:54.334635 6928 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:37:54.335313 6928 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:37:54.335313 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9 I1226 10:37:54.335990 6928 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount E1226 10:37:58.344499 6928 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#sharename-pvc-69pt7##pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9#default) mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:37:58.898871 6928 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:37:58.898871 6928 utils.go:77] GRPC request: {} I1226 10:37:58.898871 6928 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:37:58.901944 6928 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:37:58.901986 6928 utils.go:77] GRPC request: {} I1226 10:37:58.901986 6928 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 8 lines ... I1226 10:37:58.906220 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:37:58.907184 6928 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:37:58.907369 6928 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:37:58.907580 6928 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:37:58.907580 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9 I1226 10:37:58.908418 6928 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount E1226 10:38:00.512907 6928 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#sharename-pvc-69pt7##pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9#default) mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:38:01.603462 6928 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:38:01.603462 6928 utils.go:77] GRPC request: {} I1226 10:38:01.603462 6928 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:38:01.605944 6928 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:38:01.605944 6928 utils.go:77] GRPC request: {} I1226 10:38:01.605944 6928 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 11 lines ... I1226 10:38:01.610539 6928 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:01.610539 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9 I1226 10:38:01.611358 6928 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:03.048322 6928 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:38:03.048322 6928 utils.go:77] GRPC request: {} I1226 10:38:03.048322 6928 utils.go:83] GRPC response: {"ready":{"value":true}} E1226 10:38:03.244744 6928 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#sharename-pvc-69pt7##pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9#default) mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:38:05.319043 6928 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:38:05.319043 6928 utils.go:77] GRPC request: {} I1226 10:38:05.319043 6928 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:38:05.321718 6928 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:38:05.321718 6928 utils.go:77] GRPC request: {} I1226 10:38:05.321718 6928 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 8 lines ... I1226 10:38:05.325629 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:05.326279 6928 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:05.326279 6928 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:05.327045 6928 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:05.327045 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9 I1226 10:38:05.327564 6928 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount E1226 10:38:06.924209 6928 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#sharename-pvc-69pt7##pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9#default) mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:38:11.007389 6928 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:38:11.007389 6928 utils.go:77] GRPC request: {} I1226 10:38:11.007904 6928 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:38:11.010036 6928 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:38:11.010036 6928 utils.go:77] GRPC request: {} I1226 10:38:11.010036 6928 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 8 lines ... I1226 10:38:11.015684 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:11.016372 6928 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:11.016372 6928 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:11.016962 6928 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:11.016962 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9 I1226 10:38:11.017660 6928 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount E1226 10:38:12.623017 6928 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#sharename-pvc-69pt7##pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9#default) mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:38:20.722540 6928 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:38:20.722540 6928 utils.go:77] GRPC request: {} I1226 10:38:20.722875 6928 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:38:20.725072 6928 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:38:20.725072 6928 utils.go:77] GRPC request: {} I1226 10:38:20.725072 6928 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 8 lines ... I1226 10:38:20.733248 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:20.733248 6928 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:20.733248 6928 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:20.733829 6928 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:20.733829 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9 I1226 10:38:20.734625 6928 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount E1226 10:38:22.362703 6928 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#sharename-pvc-69pt7##pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9#default) mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:38:33.052007 6928 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:38:33.052007 6928 utils.go:77] GRPC request: {} I1226 10:38:33.052007 6928 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:38:38.403353 6928 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:38:38.403876 6928 utils.go:77] GRPC request: {} I1226 10:38:38.403876 6928 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 11 lines ... I1226 10:38:38.410637 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:38.410637 6928 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:38.410637 6928 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:38.411315 6928 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:38:38.411315 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9 I1226 10:38:38.412008 6928 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount E1226 10:38:40.044424 6928 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#sharename-pvc-69pt7##pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9#default) mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:39:03.049876 6928 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:39:03.049876 6928 utils.go:77] GRPC request: {} I1226 10:39:03.049876 6928 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:39:12.086443 6928 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:39:12.086443 6928 utils.go:77] GRPC request: {} I1226 10:39:12.086443 6928 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 11 lines ... I1226 10:39:12.093718 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:39:12.093718 6928 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:39:12.094257 6928 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:39:12.094415 6928 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:39:12.094415 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9 I1226 10:39:12.095167 6928 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount E1226 10:39:13.805790 6928 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#sharename-pvc-69pt7##pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9#default) mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:39:33.043087 6928 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:39:33.043614 6928 utils.go:77] GRPC request: {} I1226 10:39:33.043702 6928 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:40:03.049794 6928 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:40:03.049794 6928 utils.go:77] GRPC request: {} I1226 10:40:03.049794 6928 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 14 lines ... I1226 10:40:17.841698 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:40:17.841698 6928 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:40:17.841698 6928 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:40:17.842363 6928 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:40:17.842363 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9 I1226 10:40:17.843292 6928 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount E1226 10:40:19.485143 6928 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#sharename-pvc-69pt7##pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9#default) mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:40:33.045233 6928 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:40:33.045233 6928 utils.go:77] GRPC request: {} I1226 10:40:33.045233 6928 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:41:03.042514 6928 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:41:03.042514 6928 utils.go:77] GRPC request: {} I1226 10:41:03.042514 6928 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 20 lines ... I1226 10:42:21.509872 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:42:21.510746 6928 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:42:21.510746 6928 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:42:21.510923 6928 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:42:21.510923 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9 I1226 10:42:21.511631 6928 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount E1226 10:42:23.198690 6928 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#sharename-pvc-69pt7##pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9#default) mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:42:33.049342 6928 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:42:33.049342 6928 utils.go:77] GRPC request: {} I1226 10:42:33.049342 6928 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:43:03.052939 6928 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:43:03.052939 6928 utils.go:77] GRPC request: {} I1226 10:43:03.052939 6928 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 20 lines ... I1226 10:44:25.318655 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:44:25.319341 6928 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:44:25.319341 6928 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:44:25.320229 6928 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount I1226 10:44:25.320229 6928 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9 I1226 10:44:25.320756 6928 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount E1226 10:44:26.994160 6928 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#sharename-pvc-69pt7##pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9#default) mount \\f2d382de91e544306a75352.file.core.windows.net\sharename-pvc-69pt7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-9bbf7b63-030e-4999-a5a2-6bf849a929a9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:44:33.050484 6928 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:44:33.050484 6928 utils.go:77] GRPC request: {} I1226 10:44:33.050484 6928 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:45:01.525592 6928 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:45:01.525592 6928 utils.go:77] GRPC request: {} I1226 10:45:01.525868 6928 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 289 lines ... Streaming logs below: I1226 10:12:51.502995 7060 azurefile.go:299] driver userAgent: file.csi.azure.com/e2e-e60d471c240439ae2631573c35838e7a7cb19931 e2e-test I1226 10:12:51.529901 7060 azure.go:65] set QPS(25.000000) and QPS Burst(50) for driver kubeClient I1226 10:12:51.536436 7060 azure.go:80] reading cloud config from secret kube-system/azure-cloud-provider I1226 10:12:51.573644 7060 round_trippers.go:553] GET https://10.0.0.1:443/api/v1/namespaces/kube-system/secrets/azure-cloud-provider 404 Not Found in 33 milliseconds I1226 10:12:51.574901 7060 azure.go:87] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I1226 10:12:51.574901 7060 azure.go:92] could not read cloud config from secret kube-system/azure-cloud-provider I1226 10:12:51.574901 7060 azure.go:102] use default AZURE_CREDENTIAL_FILE env var: C:\k\azure.json I1226 10:12:51.582294 7060 azure.go:110] read cloud config from file: C:\k\azure.json successfully I1226 10:12:51.585391 7060 azure_auth.go:245] Using AzurePublicCloud environment I1226 10:12:51.586016 7060 azure_auth.go:130] azure: using client_id+client_secret to retrieve access token I1226 10:12:51.587211 7060 azure_diskclient.go:68] Azure DisksClient using API version: 2022-03-02 ... skipping 40 lines ... I1226 10:13:57.126866 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:13:57.126866 7060 utils.go:77] GRPC request: {} I1226 10:13:57.126866 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:13:57.128793 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeStageVolume I1226 10:13:57.128793 7060 utils.go:77] GRPC request: {"staging_target_path":"\\var\\lib\\kubelet\\plugins\\kubernetes.io\\csi\\pv\\azurefile-2540-file.csi.azure.com-preprovsioned-pv-6kr5h\\globalmount","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_id":"kubetest-vn9xafm4#f4f4b9e44678c403d89fc3a#pre-provisioned-multiple-pods##pre-provisioned-multiple-pods#azurefile-25401"} I1226 10:13:57.136168 7060 round_trippers.go:553] GET https://10.0.0.1:443/api/v1/namespaces/azurefile-25401/secrets/azure-storage-account-f4f4b9e44678c403d89fc3a-secret 404 Not Found in 6 milliseconds W1226 10:13:57.136429 7060 azurefile.go:668] GetStorageAccountFromSecret(azure-storage-account-f4f4b9e44678c403d89fc3a-secret, azurefile-25401) failed with error: could not get secret(azure-storage-account-f4f4b9e44678c403d89fc3a-secret): secrets "azure-storage-account-f4f4b9e44678c403d89fc3a-secret" not found I1226 10:13:57.136429 7060 azurefile.go:670] use cluster identity to get account key from (0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e, kubetest-vn9xafm4, f4f4b9e44678c403d89fc3a) (2022-12-26T10:13:57.1364290Z) INFO: REQUEST: POST https://login.microsoftonline.com/097f89a0-9286-43d2-9a1a-08f1d49b1af8/oauth2/token User-Agent: Go/go1.19.4 (amd64-windows) go-autorest/adal/v1.0.0 Content-Type: application/x-www-form-urlencoded (2022-12-26T10:13:57.5081605Z) INFO: RESPONSE: 200 https://login.microsoftonline.com/097f89a0-9286-43d2-9a1a-08f1d49b1af8/oauth2/token Cache-Control: no-store, no-cache ... skipping 352 lines ... I1226 10:22:02.973087 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:02.973806 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:02.973806 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:02.974423 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:02.974423 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 I1226 10:22:02.974423 7060 safe_mounter_windows.go:97] begin to mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount E1226 10:22:04.653685 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f38065232573a4f48bbfbc8#fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9###kube-system) mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:22:05.193559 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:22:05.193559 7060 utils.go:77] GRPC request: {} I1226 10:22:05.194060 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:22:05.196656 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:22:05.196656 7060 utils.go:77] GRPC request: {} I1226 10:22:05.196656 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 8 lines ... I1226 10:22:05.200734 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:05.200734 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:05.200734 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:05.201433 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:05.201433 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 I1226 10:22:05.201964 7060 safe_mounter_windows.go:97] begin to mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount E1226 10:22:06.771392 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f38065232573a4f48bbfbc8#fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9###kube-system) mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:22:07.841979 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:22:07.841979 7060 utils.go:77] GRPC request: {} I1226 10:22:07.842187 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:22:07.844294 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:22:07.844294 7060 utils.go:77] GRPC request: {} I1226 10:22:07.844294 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 8 lines ... I1226 10:22:07.848754 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:07.849459 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:07.849479 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:07.849630 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:07.849630 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 I1226 10:22:07.850368 7060 safe_mounter_windows.go:97] begin to mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount E1226 10:22:09.417522 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f38065232573a4f48bbfbc8#fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9###kube-system) mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:22:11.443484 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:22:11.443484 7060 utils.go:77] GRPC request: {} I1226 10:22:11.444128 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:22:11.446292 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:22:11.446292 7060 utils.go:77] GRPC request: {} I1226 10:22:11.446292 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 8 lines ... I1226 10:22:11.451117 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:11.451176 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:11.451176 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:11.451841 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:11.451841 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 I1226 10:22:11.452552 7060 safe_mounter_windows.go:97] begin to mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount E1226 10:22:13.091966 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f38065232573a4f48bbfbc8#fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9###kube-system) mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:22:17.206857 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:22:17.206857 7060 utils.go:77] GRPC request: {} I1226 10:22:17.206857 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:22:17.209649 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:22:17.209649 7060 utils.go:77] GRPC request: {} I1226 10:22:17.209649 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 8 lines ... I1226 10:22:17.214682 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:17.214682 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:17.214682 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:17.215386 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:17.215386 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 I1226 10:22:17.216077 7060 safe_mounter_windows.go:97] begin to mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount E1226 10:22:18.803159 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f38065232573a4f48bbfbc8#fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9###kube-system) mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:22:26.902336 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:22:26.902336 7060 utils.go:77] GRPC request: {} I1226 10:22:26.902336 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:22:26.904487 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:22:26.904487 7060 utils.go:77] GRPC request: {} I1226 10:22:26.904487 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 8 lines ... I1226 10:22:26.908942 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:26.909590 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:26.909590 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:26.909590 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:26.909590 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 I1226 10:22:26.910281 7060 safe_mounter_windows.go:97] begin to mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount E1226 10:22:28.542293 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f38065232573a4f48bbfbc8#fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9###kube-system) mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:22:32.746734 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:22:32.746734 7060 utils.go:77] GRPC request: {} I1226 10:22:32.746734 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:22:44.552551 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:22:44.552551 7060 utils.go:77] GRPC request: {} I1226 10:22:44.552626 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 11 lines ... I1226 10:22:44.558553 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:44.559179 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:44.559179 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:44.559316 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:22:44.559316 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 I1226 10:22:44.559907 7060 safe_mounter_windows.go:97] begin to mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount E1226 10:22:46.141035 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f38065232573a4f48bbfbc8#fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9###kube-system) mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:23:02.747564 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:23:02.747564 7060 utils.go:77] GRPC request: {} I1226 10:23:02.748206 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:23:18.247181 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:23:18.247181 7060 utils.go:77] GRPC request: {} I1226 10:23:18.247708 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 11 lines ... I1226 10:23:18.254422 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:23:18.254422 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:23:18.254422 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:23:18.255292 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:23:18.255292 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 I1226 10:23:18.255292 7060 safe_mounter_windows.go:97] begin to mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount E1226 10:23:19.872155 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f38065232573a4f48bbfbc8#fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9###kube-system) mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:23:32.741948 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:23:32.741948 7060 utils.go:77] GRPC request: {} I1226 10:23:32.741948 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:24:02.741964 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:24:02.741964 7060 utils.go:77] GRPC request: {} I1226 10:24:02.741964 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 14 lines ... I1226 10:24:23.934465 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:24:23.934523 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:24:23.934523 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:24:23.935077 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:24:23.935077 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 I1226 10:24:23.935757 7060 safe_mounter_windows.go:97] begin to mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount E1226 10:24:27.850027 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f38065232573a4f48bbfbc8#fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9###kube-system) mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:24:32.741498 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:24:32.741498 7060 utils.go:77] GRPC request: {} I1226 10:24:32.741631 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:25:02.737697 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:25:02.737697 7060 utils.go:77] GRPC request: {} I1226 10:25:02.737697 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 20 lines ... I1226 10:26:29.893810 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:26:29.893868 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:26:29.893868 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:26:29.894588 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:26:29.894588 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 I1226 10:26:29.895222 7060 safe_mounter_windows.go:97] begin to mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount E1226 10:26:31.570482 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f38065232573a4f48bbfbc8#fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9###kube-system) mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:26:32.738895 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:26:32.738895 7060 utils.go:77] GRPC request: {} I1226 10:26:32.738895 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:27:02.741250 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:27:02.741250 7060 utils.go:77] GRPC request: {} I1226 10:27:02.741250 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 23 lines ... I1226 10:28:33.614030 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:28:33.614622 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:28:33.614622 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:28:33.615098 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:28:33.615098 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 I1226 10:28:33.615098 7060 safe_mounter_windows.go:97] begin to mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount E1226 10:28:35.284412 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f38065232573a4f48bbfbc8#fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9###kube-system) mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:29:02.740847 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:29:02.740847 7060 utils.go:77] GRPC request: {} I1226 10:29:02.740847 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:29:32.740587 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:29:32.740587 7060 utils.go:77] GRPC request: {} I1226 10:29:32.740587 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 20 lines ... I1226 10:30:37.357134 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:30:37.358089 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:30:37.358144 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:30:37.358347 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:30:37.358347 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 I1226 10:30:37.359119 7060 safe_mounter_windows.go:97] begin to mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount E1226 10:30:39.056958 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f38065232573a4f48bbfbc8#fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9###kube-system) mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:31:02.751354 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:31:02.751354 7060 utils.go:77] GRPC request: {} I1226 10:31:02.751354 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:31:32.753449 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:31:32.753449 7060 utils.go:77] GRPC request: {} I1226 10:31:32.753449 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 20 lines ... I1226 10:32:41.076424 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:32:41.077112 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:32:41.077112 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:32:41.077248 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:32:41.077248 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 I1226 10:32:41.078025 7060 safe_mounter_windows.go:97] begin to mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount E1226 10:32:42.780708 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f38065232573a4f48bbfbc8#fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9###kube-system) mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:33:02.748243 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:33:02.748243 7060 utils.go:77] GRPC request: {} I1226 10:33:02.748243 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:33:32.754975 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:33:32.754975 7060 utils.go:77] GRPC request: {} I1226 10:33:32.755059 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 20 lines ... I1226 10:34:44.852535 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:34:44.852535 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:34:44.852535 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:34:44.853108 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:34:44.853108 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 I1226 10:34:44.853798 7060 safe_mounter_windows.go:97] begin to mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount E1226 10:34:48.889807 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f38065232573a4f48bbfbc8#fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9###kube-system) mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:35:02.756521 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:35:02.756521 7060 utils.go:77] GRPC request: {} I1226 10:35:02.756521 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:35:32.745104 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:35:32.745104 7060 utils.go:77] GRPC request: {} I1226 10:35:32.745104 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 20 lines ... I1226 10:36:50.932783 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:36:50.933387 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:36:50.933432 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:36:50.933552 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount I1226 10:36:50.933552 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 I1226 10:36:50.934408 7060 safe_mounter_windows.go:97] begin to mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount E1226 10:36:52.608777 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f38065232573a4f48bbfbc8#fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9###kube-system) mount \\f38065232573a4f48bbfbc8.file.core.windows.net\fileprefix-pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-93755d07-c652-48ab-a3ef-5c4d8d23fff9\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:37:02.756871 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:37:02.756871 7060 utils.go:77] GRPC request: {} I1226 10:37:02.756871 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:37:32.759990 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:37:32.759990 7060 utils.go:77] GRPC request: {} I1226 10:37:32.759990 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 135 lines ... I1226 10:46:27.663707 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:27.663707 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:27.663707 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:27.664403 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:27.664403 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 I1226 10:46:27.665101 7060 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount E1226 10:46:29.319785 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7###azurefile-2546) mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:46:29.910003 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:46:29.910003 7060 utils.go:77] GRPC request: {} I1226 10:46:29.910401 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:46:29.912726 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:46:29.912726 7060 utils.go:77] GRPC request: {} I1226 10:46:29.912726 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 8 lines ... I1226 10:46:29.917068 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:29.917693 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:29.917744 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:29.917833 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:29.917833 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 I1226 10:46:29.918468 7060 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount E1226 10:46:31.554352 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7###azurefile-2546) mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:46:32.611669 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:46:32.611669 7060 utils.go:77] GRPC request: {} I1226 10:46:32.612190 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:46:32.613929 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:46:32.614478 7060 utils.go:77] GRPC request: {} I1226 10:46:32.614478 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 11 lines ... I1226 10:46:32.619631 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:32.619631 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 I1226 10:46:32.620362 7060 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:32.754685 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:46:32.754685 7060 utils.go:77] GRPC request: {} I1226 10:46:32.754685 7060 utils.go:83] GRPC response: {"ready":{"value":true}} E1226 10:46:34.239409 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7###azurefile-2546) mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:46:36.292330 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:46:36.292330 7060 utils.go:77] GRPC request: {} I1226 10:46:36.292330 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:46:36.294908 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:46:36.294908 7060 utils.go:77] GRPC request: {} I1226 10:46:36.294908 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 8 lines ... I1226 10:46:36.299121 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:36.299643 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:36.299643 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:36.300165 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:36.300165 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 I1226 10:46:36.300704 7060 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount E1226 10:46:37.881784 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7###azurefile-2546) mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:46:41.912571 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:46:41.912571 7060 utils.go:77] GRPC request: {} I1226 10:46:41.912571 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:46:41.915045 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:46:41.915045 7060 utils.go:77] GRPC request: {} I1226 10:46:41.915045 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 8 lines ... I1226 10:46:41.920117 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:41.920767 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:41.920824 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:41.921002 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:41.921002 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 I1226 10:46:41.921770 7060 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount E1226 10:46:43.599141 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7###azurefile-2546) mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:46:51.702156 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:46:51.702156 7060 utils.go:77] GRPC request: {} I1226 10:46:51.702156 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} I1226 10:46:51.706709 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:46:51.706709 7060 utils.go:77] GRPC request: {} I1226 10:46:51.706709 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 8 lines ... I1226 10:46:51.710934 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:51.710992 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:51.710992 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:51.711990 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:46:51.711990 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 I1226 10:46:51.712124 7060 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount E1226 10:46:53.384253 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7###azurefile-2546) mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:47:02.765350 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:47:02.765393 7060 utils.go:77] GRPC request: {} I1226 10:47:02.765439 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:47:09.418334 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:47:09.418334 7060 utils.go:77] GRPC request: {} I1226 10:47:09.418627 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 11 lines ... I1226 10:47:09.424826 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:47:09.424826 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:47:09.424826 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:47:09.425594 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:47:09.425594 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 I1226 10:47:09.426333 7060 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount E1226 10:47:11.043576 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7###azurefile-2546) mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:47:32.765242 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:47:32.765771 7060 utils.go:77] GRPC request: {} I1226 10:47:32.765892 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:47:43.143483 7060 utils.go:76] GRPC call: /csi.v1.Node/NodeGetCapabilities I1226 10:47:43.143483 7060 utils.go:77] GRPC request: {} I1226 10:47:43.143483 7060 utils.go:83] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":2}}}]} ... skipping 11 lines ... I1226 10:47:43.150048 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:47:43.150048 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:47:43.150588 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:47:43.150748 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:47:43.150748 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 I1226 10:47:43.151593 7060 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount E1226 10:47:47.809480 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7###azurefile-2546) mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:48:02.763677 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:48:02.763677 7060 utils.go:77] GRPC request: {} I1226 10:48:02.763677 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:48:32.755511 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:48:32.755511 7060 utils.go:77] GRPC request: {} I1226 10:48:32.756121 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 14 lines ... I1226 10:48:51.887886 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:48:51.887886 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:48:51.887886 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:48:51.889348 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:48:51.889348 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 I1226 10:48:51.889348 7060 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount E1226 10:48:53.568648 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7###azurefile-2546) mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:49:02.766050 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:49:02.766050 7060 utils.go:77] GRPC request: {} I1226 10:49:02.766050 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:49:32.763917 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:49:32.763917 7060 utils.go:77] GRPC request: {} I1226 10:49:32.763917 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 20 lines ... I1226 10:50:55.675580 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:50:55.675810 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:50:55.675810 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:50:55.676504 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:50:55.676504 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 I1226 10:50:55.676504 7060 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount E1226 10:50:57.362311 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7###azurefile-2546) mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:51:02.766854 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:51:02.766854 7060 utils.go:77] GRPC request: {} I1226 10:51:02.766939 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:51:32.772343 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:51:32.772343 7060 utils.go:77] GRPC request: {} I1226 10:51:32.772343 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 20 lines ... I1226 10:52:59.399520 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:52:59.400442 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:52:59.400442 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:52:59.401043 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:52:59.401075 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 I1226 10:52:59.401203 7060 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount E1226 10:53:01.048512 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7###azurefile-2546) mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:53:02.759355 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:53:02.759355 7060 utils.go:77] GRPC request: {} I1226 10:53:02.759355 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:53:32.766970 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:53:32.766970 7060 utils.go:77] GRPC request: {} I1226 10:53:32.766970 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 23 lines ... I1226 10:55:03.084879 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:55:03.085586 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:55:03.085586 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:55:03.086299 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:55:03.086343 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 I1226 10:55:03.086740 7060 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount E1226 10:55:04.724025 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7###azurefile-2546) mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:55:32.760726 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:55:32.760726 7060 utils.go:77] GRPC request: {} I1226 10:55:32.760726 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:56:02.765858 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:56:02.765858 7060 utils.go:77] GRPC request: {} I1226 10:56:02.765858 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 20 lines ... I1226 10:57:06.784136 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:57:06.785628 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:57:06.785628 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:57:06.785781 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:57:06.785781 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 I1226 10:57:06.786381 7060 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount E1226 10:57:10.722028 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7###azurefile-2546) mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:57:32.771339 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:57:32.771339 7060 utils.go:77] GRPC request: {} I1226 10:57:32.771339 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 10:58:02.769315 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:58:02.769315 7060 utils.go:77] GRPC request: {} I1226 10:58:02.769315 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 20 lines ... I1226 10:59:12.814838 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:59:12.815427 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:59:12.815427 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:59:12.816481 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 10:59:12.816481 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 I1226 10:59:12.816481 7060 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount E1226 10:59:14.456380 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7###azurefile-2546) mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 10:59:32.776566 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 10:59:32.776566 7060 utils.go:77] GRPC request: {} I1226 10:59:32.776566 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 11:00:02.762851 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 11:00:02.762851 7060 utils.go:77] GRPC request: {} I1226 11:00:02.762851 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 20 lines ... I1226 11:01:16.535656 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 11:01:16.536276 7060 azure_common_windows.go:73] Removing path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 11:01:16.536355 7060 safe_mounter_windows.go:133] Remove directory: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 11:01:16.536584 7060 safe_mounter_windows.go:70] SMBMount: remote path: \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 local path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount I1226 11:01:16.536584 7060 safe_mounter_windows.go:220] Exists path: \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 I1226 11:01:16.537287 7060 safe_mounter_windows.go:97] begin to mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on c:\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount E1226 11:01:18.206715 7060 utils.go:81] GRPC error: rpc error: code = Internal desc = volume(kubetest-vn9xafm4#f2d382de91e544306a75352#pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7###azurefile-2546) mount \\f2d382de91e544306a75352.file.core.windows.net\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7 on \var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-a4ff5cc1-5167-4ae9-9005-59c113628de7\globalmount failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: "New-SmbGlobalMapping : Access is denied. \r\nAt line:1 char:190\r\n+ ... ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (MSFT_SmbGlobalMapping:ROOT/Microsoft/...mbGlobalMapping) [New-SmbGlob \r\n alMapping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 5,New-SmbGlobalMapping\r\n \r\n", err: exit status 1 I1226 11:01:32.773498 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 11:01:32.773581 7060 utils.go:77] GRPC request: {} I1226 11:01:32.773581 7060 utils.go:83] GRPC response: {"ready":{"value":true}} I1226 11:02:02.777619 7060 utils.go:76] GRPC call: /csi.v1.Identity/Probe I1226 11:02:02.777619 7060 utils.go:77] GRPC request: {} I1226 11:02:02.777619 7060 utils.go:83] GRPC response: {"ready":{"value":true}} ... skipping 408 lines ... # HELP go_gc_heap_objects_objects Number of objects, live or unswept, occupying heap memory. # TYPE go_gc_heap_objects_objects gauge go_gc_heap_objects_objects 56037 # HELP go_gc_heap_tiny_allocs_objects_total Count of small allocations that are packed together into blocks. These allocations are counted separately from other allocations because each individual allocation is not tracked by the runtime, only their block. Each block is already accounted for in allocs-by-size and frees-by-size. # TYPE go_gc_heap_tiny_allocs_objects_total counter go_gc_heap_tiny_allocs_objects_total 22999 # HELP go_gc_limiter_last_enabled_gc_cycle GC cycle the last time the GC CPU limiter was enabled. This metric is useful for diagnosing the root cause of an out-of-memory error, because the limiter trades memory for CPU time when the GC's CPU time gets too high. This is most likely to occur with use of SetMemoryLimit. The first GC cycle is cycle 1, so a value of 0 indicates that it was never enabled. # TYPE go_gc_limiter_last_enabled_gc_cycle gauge go_gc_limiter_last_enabled_gc_cycle 0 # HELP go_gc_pauses_seconds Distribution individual GC-related stop-the-world pause latencies. # TYPE go_gc_pauses_seconds histogram go_gc_pauses_seconds_bucket{le="9.999999999999999e-10"} 0 go_gc_pauses_seconds_bucket{le="9.999999999999999e-09"} 0 ... skipping 267 lines ... [38;5;214mgoroutine 1953 [chan send][0m [38;5;243mgithub.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2.1()[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/github.com/onsi/ginkgo/v2/internal/suite.go:746[0m [38;5;243mpanic({0x20eb580, 0xc0004b48c0})[0m [38;5;243m/usr/local/go/src/runtime/panic.go:884[0m [38;5;243mgithub.com/onsi/ginkgo/v2.Fail({0xc000cecb80, 0x7b}, {0xc00082b6f0?, 0x228f280?, 0xc00082b710?})[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/github.com/onsi/ginkgo/v2/core_dsl.go:352[0m [38;5;243mk8s.io/kubernetes/test/e2e/framework.Fail({0xc0007e2000, 0x66}, {0xc00082b788?, 0xc0007e2000?, 0xc00082b7b0?})[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/log.go:61[0m [38;5;243mk8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0x1, {0x27a1320, 0xc0009d16b0}, {0x0?, 0xc001069d00?, 0xe?})[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/expect.go:76[0m [38;5;243mk8s.io/kubernetes/test/e2e/framework.ExpectNoError(...)[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/expect.go:43[0m [38;5;214m[1m> sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*TestPod).WaitForSuccess(0x23024db?)[0m ... skipping 3 lines ... [1m[38;5;214m> framework.ExpectNoError(err)[0m | } | [38;5;214m[1m> sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedResizeVolumeTest).Run(0xc00082bf08, {0x27d5520, 0xc000fc04e0}, 0xc0008a74a0)[0m [38;5;214m[1m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/dynamically_provisioned_resize_volume_tester.go:73[0m | defer tpod.Cleanup() | ginkgo.By("checking that the pods command exits with no error") [1m[38;5;214m> tpod.WaitForSuccess()[0m | | pvcName := tpod.pod.Spec.Volumes[0].VolumeSource.PersistentVolumeClaim.ClaimName [38;5;214m[1m> sigs.k8s.io/azurefile-csi-driver/test/e2e.glob..func1.10()[0m [38;5;214m[1m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:390[0m | StorageClassParameters: scParameters, ... skipping 8 lines ... [38;5;243mgithub.com/onsi/ginkgo/v2/internal.(*Suite).runNode[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738[0m [38;5;243m------------------------------[0m [38;5;9m[1mSummarizing 5 Failures:[0m [38;5;9m[FAIL][0m [0mDynamic Provisioning [38;5;243m[38;5;9m[1m[It] should create a volume on demand with mount options [kubernetes.io/azure-file] [file.csi.azure.com] [Windows][0m[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:221[0m [38;5;9m[FAIL][0m [0mDynamic Provisioning [38;5;243m[38;5;9m[1m[It] should create a pod with volume mount subpath [file.csi.azure.com] [Windows][0m[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:688[0m [38;5;9m[FAIL][0m [0mDynamic Provisioning [38;5;243m[38;5;9m[1m[It] should create multiple PV objects, bind to PVCs and attach all to different pods on the same node [kubernetes.io/azure-file] [file.csi.azure.com] [Windows][0m[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:693[0m [38;5;214m[TIMEDOUT][0m [0mDynamic Provisioning [38;5;243m[38;5;214m[1m[It] should create a volume on demand and resize it [kubernetes.io/azure-file] [file.csi.azure.com] [Windows][0m[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/dynamic_provisioning_test.go:361[0m [38;5;214m[TIMEDOUT][0m [0m[38;5;214m[1m[AfterSuite] [0m[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/suite_test.go:148[0m [38;5;9m[1mRan 14 of 36 Specs in 3630.113 seconds[0m [38;5;9m[1mFAIL! - Suite Timeout Elapsed[0m -- [38;5;10m[1m10 Passed[0m | [38;5;9m[1m4 Failed[0m | [38;5;11m[1m0 Pending[0m | [38;5;14m[1m22 Skipped[0m [38;5;228mYou're using deprecated Ginkgo functionality:[0m [38;5;228m=============================================[0m [38;5;11mSupport for custom reporters has been removed in V2. Please read the documentation linked to below for Ginkgo's new behavior and for a migration path:[0m [1mLearn more at:[0m [38;5;14m[4mhttps://onsi.github.io/ginkgo/MIGRATING_TO_V2#removed-custom-reporters[0m [38;5;243mTo silence deprecations that can be silenced set the following environment variable:[0m [38;5;243mACK_GINKGO_DEPRECATIONS=2.4.0[0m --- FAIL: TestE2E (3630.11s) FAIL release "azurefile-csi-driver" uninstalled make[1]: Leaving directory '/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver' FAIL sigs.k8s.io/azurefile-csi-driver/test/e2e 3630.172s FAIL make: *** [Makefile:85: e2e-test] Error 1 2022/12/26 11:02:21 process.go:155: Step 'make e2e-test' finished in 1h2m21.619956845s 2022/12/26 11:02:21 aksengine_helpers.go:425: downloading /root/tmp3171201368/log-dump.sh from https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh 2022/12/26 11:02:21 util.go:70: curl https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh 2022/12/26 11:02:21 process.go:153: Running: chmod +x /root/tmp3171201368/log-dump.sh 2022/12/26 11:02:21 process.go:155: Step 'chmod +x /root/tmp3171201368/log-dump.sh' finished in 3.366244ms 2022/12/26 11:02:21 aksengine_helpers.go:425: downloading /root/tmp3171201368/log-dump-daemonset.yaml from https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump-daemonset.yaml ... skipping 110 lines ... Flags -d Detailed option, when used with list, dumps the json of the object Write-Progress : Win32 internal error "Access is denied" 0x5 occurred while reading the console output buffer. Contact Microsoft Customer Support Services. At C:\windows\system32\WindowsPowerShell\v1.0\Modules\Microsoft.PowerShell.Archive\Microsoft.PowerShell.Archive.psm1:1132 char:9 + Write-Progress -Activity $cmdletName -Status $status -Percent ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ... skipping 53 lines ... Flags -d Detailed option, when used with list, dumps the json of the object Write-Progress : Win32 internal error "Access is denied" 0x5 occurred while reading the console output buffer. Contact Microsoft Customer Support Services. At C:\windows\system32\WindowsPowerShell\v1.0\Modules\Microsoft.PowerShell.Archive\Microsoft.PowerShell.Archive.psm1:1132 char:9 + Write-Progress -Activity $cmdletName -Status $status -Percent ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ... skipping 8 lines ... Warning: Permanently added '6939k8s001' (ECDSA) to the list of known hosts. Collecting logs for vm 6939k8s002 Testing connection to host 6939k8s002. Warning: Permanently added 'kubetest-vn9xafm4.uksouth.cloudapp.azure.com,20.26.169.59' (ECDSA) to the list of known hosts. Authorized uses only. All activity may be monitored and reported. channel 0: open failed: connect failed: Name or service not known stdio forwarding failed ssh_exchange_identification: Connection closed by remote host 2022/12/26 11:04:03 process.go:155: Step 'bash -c /root/tmp3171201368/win-ci-logs-collector.sh kubetest-vn9xafm4.uksouth.cloudapp.azure.com /root/tmp3171201368 /etc/ssh-key-secret/ssh-private' finished in 1m18.157307138s 2022/12/26 11:04:03 aksengine.go:1141: Deleting resource group: kubetest-vn9xafm4. 2022/12/26 11:10:13 process.go:96: Saved XML output to /logs/artifacts/junit_runner.xml. 2022/12/26 11:10:13 process.go:153: Running: bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}" 2022/12/26 11:10:15 process.go:155: Step 'bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}"' finished in 1.719735658s 2022/12/26 11:10:15 main.go:328: Something went wrong: encountered 1 errors: [error during make e2e-test: exit status 2] + EXIT_VALUE=1 + set +o xtrace Cleaning up after docker in docker. ================================================================================ Cleaning up after docker f11a1fab85ff ... skipping 4 lines ...