This job view page is being replaced by Spyglass soon. Check out the new job view.
PRandyzhangx: feat: switch back to csi-proxy v1beta interface on Windows
ResultABORTED
Tests 0 failed / 0 succeeded
Started2021-09-29 14:57
Elapsed49m58s
Revision4be9f85bea3b5d9175d6605f15e2608b93cf4df8
Refs 1034

No Test Failures!


Error lines from build-log.txt

... skipping 222 lines ...

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 11248  100 11248    0     0   177k      0 --:--:-- --:--:-- --:--:--  177k
Downloading https://get.helm.sh/helm-v3.7.0-linux-amd64.tar.gz
Verifying checksum... Done.
Preparing to install helm into /usr/local/bin
helm installed into /usr/local/bin/helm
docker pull k8sprow.azurecr.io/azuredisk-csi:v1.8.0-48d7bd18383c09b02fba2b10739a91587fbfde8c || make container-all push-manifest
Error response from daemon: manifest for k8sprow.azurecr.io/azuredisk-csi:v1.8.0-48d7bd18383c09b02fba2b10739a91587fbfde8c not found: manifest unknown: manifest tagged by "v1.8.0-48d7bd18383c09b02fba2b10739a91587fbfde8c" is not found
make[2]: Entering directory '/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver'
CGO_ENABLED=0 GOOS=windows go build -a -ldflags "-X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.driverVersion=v1.8.0-48d7bd18383c09b02fba2b10739a91587fbfde8c -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.gitCommit=48d7bd18383c09b02fba2b10739a91587fbfde8c -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.buildDate=2021-09-29T15:03:09Z -extldflags "-static""  -mod vendor -o _output/amd64/azurediskplugin.exe ./pkg/azurediskplugin
docker buildx rm container-builder || true
error: no builder "container-builder" found
docker buildx create --use --name=container-builder
container-builder
# enable qemu for arm64 build
# https://github.com/docker/buildx/issues/464#issuecomment-741507760
docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-aarch64
Unable to find image 'tonistiigi/binfmt:latest' locally
... skipping 1685 lines ...
                    type: string
                type: object
                oneOf:
                - required: ["persistentVolumeClaimName"]
                - required: ["volumeSnapshotContentName"]
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 60 lines ...
                    type: string
                  volumeSnapshotContentName:
                    description: volumeSnapshotContentName specifies the name of a pre-existing VolumeSnapshotContent object representing an existing volume snapshot. This field should be set if the snapshot already exists and only needs a representation in Kubernetes. This field is immutable.
                    type: string
                type: object
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 248 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 108 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 856 lines ...
        - name: csi-resizer
          image: "mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0"
          args:
            - "-csi-address=$(ADDRESS)"
            - "-v=2"
            - "-leader-election"
            - '-handle-volume-inuse-error=true'
          env:
            - name: ADDRESS
              value: /csi/csi.sock
          volumeMounts:
            - name: socket-dir
              mountPath: /csi
... skipping 368 lines ...
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Sep 29 15:13:29.704: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volumelimits-9558" for this suite.

•
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits should verify that all csinodes have volume limits","total":15,"completed":1,"skipped":2,"failed":0}

SSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (immediate binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (immediate binding)] topology
... skipping 2 lines ...
Sep 29 15:13:28.907: INFO: >>> kubeConfig: /root/tmp2380974403/kubeconfig/kubeconfig.westeurope.json
STEP: Building a namespace api object, basename topology
W0929 15:13:29.373178   34728 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
Sep 29 15:13:29.373: INFO: Found PodSecurityPolicies; testing pod creation to see if PodSecurityPolicy is enabled
Sep 29 15:13:29.485: INFO: No PSP annotation exists on dry run pod; assuming PodSecurityPolicy is disabled
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail to schedule a pod which has topologies that conflict with AllowedTopologies
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192
Sep 29 15:13:29.705: INFO: Driver didn't provide topology keys -- skipping
[AfterEach] [Testpattern: Dynamic PV (immediate binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Sep 29 15:13:29.706: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "topology-7097" for this suite.


S [SKIPPING] [1.130 seconds]
External Storage [Driver: test.csi.azure.com]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (immediate binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to schedule a pod which has topologies that conflict with AllowedTopologies [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192

    Driver didn't provide topology keys -- skipping

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:124
------------------------------
... skipping 90 lines ...
[BeforeEach] [Testpattern: Dynamic PV (delayed binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Sep 29 15:13:31.142: INFO: >>> kubeConfig: /root/tmp2380974403/kubeconfig/kubeconfig.westeurope.json
STEP: Building a namespace api object, basename topology
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail to schedule a pod which has topologies that conflict with AllowedTopologies
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192
Sep 29 15:13:31.711: INFO: Driver didn't provide topology keys -- skipping
[AfterEach] [Testpattern: Dynamic PV (delayed binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Sep 29 15:13:31.712: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "topology-8656" for this suite.


S [SKIPPING] [0.796 seconds]
External Storage [Driver: test.csi.azure.com]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (delayed binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to schedule a pod which has topologies that conflict with AllowedTopologies [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192

    Driver didn't provide topology keys -- skipping

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:124
------------------------------
... skipping 82 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)] volumeMode
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should not mount / map unused volumes in a pod [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:351
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] volumeMode should not mount / map unused volumes in a pod [LinuxOnly]","total":18,"completed":1,"skipped":329,"failed":0}

SSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (ext4)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:14:37.253: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 37 lines ...
Sep 29 15:13:31.274: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comlmjhv] to have phase Bound
Sep 29 15:13:31.385: INFO: PersistentVolumeClaim test.csi.azure.comlmjhv found but phase is Pending instead of Bound.
Sep 29 15:13:33.496: INFO: PersistentVolumeClaim test.csi.azure.comlmjhv found but phase is Pending instead of Bound.
Sep 29 15:13:35.608: INFO: PersistentVolumeClaim test.csi.azure.comlmjhv found and phase=Bound (4.334197394s)
STEP: Creating pod pod-subpath-test-dynamicpv-jbpp
STEP: Creating a pod to test subpath
Sep 29 15:13:35.942: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-jbpp" in namespace "provisioning-8114" to be "Succeeded or Failed"
Sep 29 15:13:36.052: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Pending", Reason="", readiness=false. Elapsed: 109.51945ms
Sep 29 15:13:38.165: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Pending", Reason="", readiness=false. Elapsed: 2.222568066s
Sep 29 15:13:40.279: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Pending", Reason="", readiness=false. Elapsed: 4.337024046s
Sep 29 15:13:42.396: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Pending", Reason="", readiness=false. Elapsed: 6.453521115s
Sep 29 15:13:44.507: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Pending", Reason="", readiness=false. Elapsed: 8.564725279s
Sep 29 15:13:46.618: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Pending", Reason="", readiness=false. Elapsed: 10.676053443s
... skipping 3 lines ...
Sep 29 15:13:55.064: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Pending", Reason="", readiness=false. Elapsed: 19.122245787s
Sep 29 15:13:57.174: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Pending", Reason="", readiness=false. Elapsed: 21.232103866s
Sep 29 15:13:59.287: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Pending", Reason="", readiness=false. Elapsed: 23.344429041s
Sep 29 15:14:01.438: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Pending", Reason="", readiness=false. Elapsed: 25.495474222s
Sep 29 15:14:03.549: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Succeeded", Reason="", readiness=false. Elapsed: 27.607062706s
STEP: Saw pod success
Sep 29 15:14:03.549: INFO: Pod "pod-subpath-test-dynamicpv-jbpp" satisfied condition "Succeeded or Failed"
Sep 29 15:14:03.672: INFO: Trying to get logs from node k8s-agentpool1-18436497-vmss000000 pod pod-subpath-test-dynamicpv-jbpp container test-container-subpath-dynamicpv-jbpp: <nil>
STEP: delete the pod
Sep 29 15:14:03.964: INFO: Waiting for pod pod-subpath-test-dynamicpv-jbpp to disappear
Sep 29 15:14:04.074: INFO: Pod pod-subpath-test-dynamicpv-jbpp no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-jbpp
Sep 29 15:14:04.074: INFO: Deleting pod "pod-subpath-test-dynamicpv-jbpp" in namespace "provisioning-8114"
STEP: Creating pod pod-subpath-test-dynamicpv-jbpp
STEP: Creating a pod to test subpath
Sep 29 15:14:04.299: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-jbpp" in namespace "provisioning-8114" to be "Succeeded or Failed"
Sep 29 15:14:04.409: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Pending", Reason="", readiness=false. Elapsed: 109.793134ms
Sep 29 15:14:06.521: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Pending", Reason="", readiness=false. Elapsed: 2.222072075s
Sep 29 15:14:08.633: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Pending", Reason="", readiness=false. Elapsed: 4.333688798s
Sep 29 15:14:10.745: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Pending", Reason="", readiness=false. Elapsed: 6.446108108s
Sep 29 15:14:12.856: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Pending", Reason="", readiness=false. Elapsed: 8.556729813s
Sep 29 15:14:14.967: INFO: Pod "pod-subpath-test-dynamicpv-jbpp": Phase="Succeeded", Reason="", readiness=false. Elapsed: 10.66797899s
STEP: Saw pod success
Sep 29 15:14:14.967: INFO: Pod "pod-subpath-test-dynamicpv-jbpp" satisfied condition "Succeeded or Failed"
Sep 29 15:14:15.078: INFO: Trying to get logs from node k8s-agentpool1-18436497-vmss000000 pod pod-subpath-test-dynamicpv-jbpp container test-container-subpath-dynamicpv-jbpp: <nil>
STEP: delete the pod
Sep 29 15:14:15.311: INFO: Waiting for pod pod-subpath-test-dynamicpv-jbpp to disappear
Sep 29 15:14:15.422: INFO: Pod pod-subpath-test-dynamicpv-jbpp no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-jbpp
Sep 29 15:14:15.422: INFO: Deleting pod "pod-subpath-test-dynamicpv-jbpp" in namespace "provisioning-8114"
... skipping 23 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support existing directories when readOnly specified in the volumeSource
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:395
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directories when readOnly specified in the volumeSource","total":19,"completed":1,"skipped":229,"failed":0}

SSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:14:57.232: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 217 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","total":12,"completed":1,"skipped":37,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:15:06.125: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 40 lines ...
Sep 29 15:13:30.229: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com7hrtl] to have phase Bound
Sep 29 15:13:30.341: INFO: PersistentVolumeClaim test.csi.azure.com7hrtl found but phase is Pending instead of Bound.
Sep 29 15:13:32.451: INFO: PersistentVolumeClaim test.csi.azure.com7hrtl found but phase is Pending instead of Bound.
Sep 29 15:13:34.561: INFO: PersistentVolumeClaim test.csi.azure.com7hrtl found and phase=Bound (4.332084771s)
STEP: Creating pod pod-subpath-test-dynamicpv-t8gl
STEP: Creating a pod to test subpath
Sep 29 15:13:34.902: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-t8gl" in namespace "provisioning-9408" to be "Succeeded or Failed"
Sep 29 15:13:35.011: INFO: Pod "pod-subpath-test-dynamicpv-t8gl": Phase="Pending", Reason="", readiness=false. Elapsed: 109.521505ms
Sep 29 15:13:37.127: INFO: Pod "pod-subpath-test-dynamicpv-t8gl": Phase="Pending", Reason="", readiness=false. Elapsed: 2.225133505s
Sep 29 15:13:39.238: INFO: Pod "pod-subpath-test-dynamicpv-t8gl": Phase="Pending", Reason="", readiness=false. Elapsed: 4.33624391s
Sep 29 15:13:41.349: INFO: Pod "pod-subpath-test-dynamicpv-t8gl": Phase="Pending", Reason="", readiness=false. Elapsed: 6.447193343s
Sep 29 15:13:43.460: INFO: Pod "pod-subpath-test-dynamicpv-t8gl": Phase="Pending", Reason="", readiness=false. Elapsed: 8.558200156s
Sep 29 15:13:45.572: INFO: Pod "pod-subpath-test-dynamicpv-t8gl": Phase="Pending", Reason="", readiness=false. Elapsed: 10.670287338s
Sep 29 15:13:47.683: INFO: Pod "pod-subpath-test-dynamicpv-t8gl": Phase="Pending", Reason="", readiness=false. Elapsed: 12.781601747s
Sep 29 15:13:49.794: INFO: Pod "pod-subpath-test-dynamicpv-t8gl": Phase="Pending", Reason="", readiness=false. Elapsed: 14.892554105s
Sep 29 15:13:51.906: INFO: Pod "pod-subpath-test-dynamicpv-t8gl": Phase="Pending", Reason="", readiness=false. Elapsed: 17.004336491s
Sep 29 15:13:54.018: INFO: Pod "pod-subpath-test-dynamicpv-t8gl": Phase="Pending", Reason="", readiness=false. Elapsed: 19.116580226s
Sep 29 15:13:56.133: INFO: Pod "pod-subpath-test-dynamicpv-t8gl": Phase="Succeeded", Reason="", readiness=false. Elapsed: 21.230976812s
STEP: Saw pod success
Sep 29 15:13:56.133: INFO: Pod "pod-subpath-test-dynamicpv-t8gl" satisfied condition "Succeeded or Failed"
Sep 29 15:13:56.244: INFO: Trying to get logs from node k8s-agentpool1-18436497-vmss000001 pod pod-subpath-test-dynamicpv-t8gl container test-container-subpath-dynamicpv-t8gl: <nil>
STEP: delete the pod
Sep 29 15:13:56.985: INFO: Waiting for pod pod-subpath-test-dynamicpv-t8gl to disappear
Sep 29 15:13:57.095: INFO: Pod pod-subpath-test-dynamicpv-t8gl no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-t8gl
Sep 29 15:13:57.095: INFO: Deleting pod "pod-subpath-test-dynamicpv-t8gl" in namespace "provisioning-9408"
... skipping 29 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support readOnly directory specified in the volumeMount
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:365
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly directory specified in the volumeMount","total":13,"completed":1,"skipped":138,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ext4)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:15:09.638: INFO: Driver test.csi.azure.com doesn't support ext4 -- skipping
... skipping 209 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should store data
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] volumes should store data","total":15,"completed":2,"skipped":14,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy 
  (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
... skipping 79 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","total":8,"completed":1,"skipped":72,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:15:55.728: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 58 lines ...
Sep 29 15:14:38.083: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com5lq8g] to have phase Bound
Sep 29 15:14:38.193: INFO: PersistentVolumeClaim test.csi.azure.com5lq8g found but phase is Pending instead of Bound.
Sep 29 15:14:40.304: INFO: PersistentVolumeClaim test.csi.azure.com5lq8g found but phase is Pending instead of Bound.
Sep 29 15:14:42.415: INFO: PersistentVolumeClaim test.csi.azure.com5lq8g found and phase=Bound (4.332102332s)
STEP: Creating pod pod-subpath-test-dynamicpv-f4tw
STEP: Creating a pod to test atomic-volume-subpath
Sep 29 15:14:42.757: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-f4tw" in namespace "provisioning-6052" to be "Succeeded or Failed"
Sep 29 15:14:42.875: INFO: Pod "pod-subpath-test-dynamicpv-f4tw": Phase="Pending", Reason="", readiness=false. Elapsed: 117.209744ms
Sep 29 15:14:44.987: INFO: Pod "pod-subpath-test-dynamicpv-f4tw": Phase="Pending", Reason="", readiness=false. Elapsed: 2.229381282s
Sep 29 15:14:47.099: INFO: Pod "pod-subpath-test-dynamicpv-f4tw": Phase="Pending", Reason="", readiness=false. Elapsed: 4.341519262s
Sep 29 15:14:49.211: INFO: Pod "pod-subpath-test-dynamicpv-f4tw": Phase="Pending", Reason="", readiness=false. Elapsed: 6.453006696s
Sep 29 15:14:51.328: INFO: Pod "pod-subpath-test-dynamicpv-f4tw": Phase="Pending", Reason="", readiness=false. Elapsed: 8.569946224s
Sep 29 15:14:53.439: INFO: Pod "pod-subpath-test-dynamicpv-f4tw": Phase="Pending", Reason="", readiness=false. Elapsed: 10.681109759s
... skipping 17 lines ...
Sep 29 15:15:31.464: INFO: Pod "pod-subpath-test-dynamicpv-f4tw": Phase="Running", Reason="", readiness=true. Elapsed: 48.706364487s
Sep 29 15:15:33.577: INFO: Pod "pod-subpath-test-dynamicpv-f4tw": Phase="Running", Reason="", readiness=true. Elapsed: 50.819028956s
Sep 29 15:15:35.688: INFO: Pod "pod-subpath-test-dynamicpv-f4tw": Phase="Running", Reason="", readiness=true. Elapsed: 52.930105867s
Sep 29 15:15:37.798: INFO: Pod "pod-subpath-test-dynamicpv-f4tw": Phase="Running", Reason="", readiness=true. Elapsed: 55.040418051s
Sep 29 15:15:39.909: INFO: Pod "pod-subpath-test-dynamicpv-f4tw": Phase="Succeeded", Reason="", readiness=false. Elapsed: 57.151389284s
STEP: Saw pod success
Sep 29 15:15:39.909: INFO: Pod "pod-subpath-test-dynamicpv-f4tw" satisfied condition "Succeeded or Failed"
Sep 29 15:15:40.026: INFO: Trying to get logs from node k8s-agentpool1-18436497-vmss000000 pod pod-subpath-test-dynamicpv-f4tw container test-container-subpath-dynamicpv-f4tw: <nil>
STEP: delete the pod
Sep 29 15:15:40.259: INFO: Waiting for pod pod-subpath-test-dynamicpv-f4tw to disappear
Sep 29 15:15:40.369: INFO: Pod pod-subpath-test-dynamicpv-f4tw no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-f4tw
Sep 29 15:15:40.369: INFO: Deleting pod "pod-subpath-test-dynamicpv-f4tw" in namespace "provisioning-6052"
... skipping 23 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support file as subpath [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:230
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support file as subpath [LinuxOnly]","total":18,"completed":2,"skipped":378,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes 
  should allow exec of files on the volume
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:196
... skipping 16 lines ...
Sep 29 15:15:06.951: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comc6b67] to have phase Bound
Sep 29 15:15:07.060: INFO: PersistentVolumeClaim test.csi.azure.comc6b67 found but phase is Pending instead of Bound.
Sep 29 15:15:09.172: INFO: PersistentVolumeClaim test.csi.azure.comc6b67 found but phase is Pending instead of Bound.
Sep 29 15:15:11.284: INFO: PersistentVolumeClaim test.csi.azure.comc6b67 found and phase=Bound (4.333114159s)
STEP: Creating pod exec-volume-test-dynamicpv-f54q
STEP: Creating a pod to test exec-volume-test
Sep 29 15:15:11.617: INFO: Waiting up to 5m0s for pod "exec-volume-test-dynamicpv-f54q" in namespace "volume-4666" to be "Succeeded or Failed"
Sep 29 15:15:11.728: INFO: Pod "exec-volume-test-dynamicpv-f54q": Phase="Pending", Reason="", readiness=false. Elapsed: 110.036975ms
Sep 29 15:15:13.840: INFO: Pod "exec-volume-test-dynamicpv-f54q": Phase="Pending", Reason="", readiness=false. Elapsed: 2.222017514s
Sep 29 15:15:15.950: INFO: Pod "exec-volume-test-dynamicpv-f54q": Phase="Pending", Reason="", readiness=false. Elapsed: 4.332356968s
Sep 29 15:15:18.061: INFO: Pod "exec-volume-test-dynamicpv-f54q": Phase="Pending", Reason="", readiness=false. Elapsed: 6.443093367s
Sep 29 15:15:20.173: INFO: Pod "exec-volume-test-dynamicpv-f54q": Phase="Pending", Reason="", readiness=false. Elapsed: 8.555092833s
Sep 29 15:15:22.284: INFO: Pod "exec-volume-test-dynamicpv-f54q": Phase="Pending", Reason="", readiness=false. Elapsed: 10.666354437s
... skipping 8 lines ...
Sep 29 15:15:41.291: INFO: Pod "exec-volume-test-dynamicpv-f54q": Phase="Pending", Reason="", readiness=false. Elapsed: 29.673860515s
Sep 29 15:15:43.404: INFO: Pod "exec-volume-test-dynamicpv-f54q": Phase="Pending", Reason="", readiness=false. Elapsed: 31.785973245s
Sep 29 15:15:45.516: INFO: Pod "exec-volume-test-dynamicpv-f54q": Phase="Pending", Reason="", readiness=false. Elapsed: 33.898418034s
Sep 29 15:15:47.627: INFO: Pod "exec-volume-test-dynamicpv-f54q": Phase="Pending", Reason="", readiness=false. Elapsed: 36.009094576s
Sep 29 15:15:49.738: INFO: Pod "exec-volume-test-dynamicpv-f54q": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.120830754s
STEP: Saw pod success
Sep 29 15:15:49.738: INFO: Pod "exec-volume-test-dynamicpv-f54q" satisfied condition "Succeeded or Failed"
Sep 29 15:15:49.850: INFO: Trying to get logs from node k8s-agentpool1-18436497-vmss000000 pod exec-volume-test-dynamicpv-f54q container exec-container-dynamicpv-f54q: <nil>
STEP: delete the pod
Sep 29 15:15:50.083: INFO: Waiting for pod exec-volume-test-dynamicpv-f54q to disappear
Sep 29 15:15:50.192: INFO: Pod exec-volume-test-dynamicpv-f54q no longer exists
STEP: Deleting pod exec-volume-test-dynamicpv-f54q
Sep 29 15:15:50.192: INFO: Deleting pod "exec-volume-test-dynamicpv-f54q" in namespace "volume-4666"
... skipping 21 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should allow exec of files on the volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:196
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should allow exec of files on the volume","total":12,"completed":2,"skipped":205,"failed":0}

SSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:16:31.921: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 37 lines ...
Sep 29 15:15:11.024: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.coml5ssg] to have phase Bound
Sep 29 15:15:11.138: INFO: PersistentVolumeClaim test.csi.azure.coml5ssg found but phase is Pending instead of Bound.
Sep 29 15:15:13.249: INFO: PersistentVolumeClaim test.csi.azure.coml5ssg found but phase is Pending instead of Bound.
Sep 29 15:15:15.360: INFO: PersistentVolumeClaim test.csi.azure.coml5ssg found and phase=Bound (4.336209641s)
STEP: Creating pod pod-subpath-test-dynamicpv-qgwp
STEP: Creating a pod to test subpath
Sep 29 15:15:15.696: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-qgwp" in namespace "provisioning-9254" to be "Succeeded or Failed"
Sep 29 15:15:15.806: INFO: Pod "pod-subpath-test-dynamicpv-qgwp": Phase="Pending", Reason="", readiness=false. Elapsed: 110.535002ms
Sep 29 15:15:17.916: INFO: Pod "pod-subpath-test-dynamicpv-qgwp": Phase="Pending", Reason="", readiness=false. Elapsed: 2.220438341s
Sep 29 15:15:20.028: INFO: Pod "pod-subpath-test-dynamicpv-qgwp": Phase="Pending", Reason="", readiness=false. Elapsed: 4.332621776s
Sep 29 15:15:22.140: INFO: Pod "pod-subpath-test-dynamicpv-qgwp": Phase="Pending", Reason="", readiness=false. Elapsed: 6.444865741s
Sep 29 15:15:24.256: INFO: Pod "pod-subpath-test-dynamicpv-qgwp": Phase="Pending", Reason="", readiness=false. Elapsed: 8.560388011s
Sep 29 15:15:26.368: INFO: Pod "pod-subpath-test-dynamicpv-qgwp": Phase="Pending", Reason="", readiness=false. Elapsed: 10.672872811s
... skipping 8 lines ...
Sep 29 15:15:45.386: INFO: Pod "pod-subpath-test-dynamicpv-qgwp": Phase="Pending", Reason="", readiness=false. Elapsed: 29.690214276s
Sep 29 15:15:47.502: INFO: Pod "pod-subpath-test-dynamicpv-qgwp": Phase="Pending", Reason="", readiness=false. Elapsed: 31.806130601s
Sep 29 15:15:49.614: INFO: Pod "pod-subpath-test-dynamicpv-qgwp": Phase="Pending", Reason="", readiness=false. Elapsed: 33.917979865s
Sep 29 15:15:51.725: INFO: Pod "pod-subpath-test-dynamicpv-qgwp": Phase="Pending", Reason="", readiness=false. Elapsed: 36.029788177s
Sep 29 15:15:53.837: INFO: Pod "pod-subpath-test-dynamicpv-qgwp": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.141085123s
STEP: Saw pod success
Sep 29 15:15:53.837: INFO: Pod "pod-subpath-test-dynamicpv-qgwp" satisfied condition "Succeeded or Failed"
Sep 29 15:15:53.947: INFO: Trying to get logs from node k8s-agentpool1-18436497-vmss000001 pod pod-subpath-test-dynamicpv-qgwp container test-container-subpath-dynamicpv-qgwp: <nil>
STEP: delete the pod
Sep 29 15:15:54.221: INFO: Waiting for pod pod-subpath-test-dynamicpv-qgwp to disappear
Sep 29 15:15:54.331: INFO: Pod pod-subpath-test-dynamicpv-qgwp no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-qgwp
Sep 29 15:15:54.331: INFO: Deleting pod "pod-subpath-test-dynamicpv-qgwp" in namespace "provisioning-9254"
... skipping 23 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support existing single file [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:219
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing single file [LinuxOnly]","total":13,"completed":2,"skipped":640,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes 
  should store data
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159
... skipping 74 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should store data
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should store data","total":8,"completed":2,"skipped":376,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ext3)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:17:27.825: INFO: Driver test.csi.azure.com doesn't support ext3 -- skipping
... skipping 92 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should be able to unmount after the subpath directory is deleted [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:445
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should be able to unmount after the subpath directory is deleted [LinuxOnly]","total":18,"completed":3,"skipped":499,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (ext4)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:17:53.379: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 92 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read-only inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read-only inline ephemeral volume","total":19,"completed":2,"skipped":389,"failed":0}

SSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:17:58.549: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 140 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:201
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]","total":15,"completed":3,"skipped":54,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath 
  should support non-existent path
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:194
... skipping 16 lines ...
Sep 29 15:16:36.998: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comxwrc5] to have phase Bound
Sep 29 15:16:37.109: INFO: PersistentVolumeClaim test.csi.azure.comxwrc5 found but phase is Pending instead of Bound.
Sep 29 15:16:39.220: INFO: PersistentVolumeClaim test.csi.azure.comxwrc5 found but phase is Pending instead of Bound.
Sep 29 15:16:41.331: INFO: PersistentVolumeClaim test.csi.azure.comxwrc5 found and phase=Bound (4.333635326s)
STEP: Creating pod pod-subpath-test-dynamicpv-dg58
STEP: Creating a pod to test subpath
Sep 29 15:16:41.665: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-dg58" in namespace "provisioning-5992" to be "Succeeded or Failed"
Sep 29 15:16:41.777: INFO: Pod "pod-subpath-test-dynamicpv-dg58": Phase="Pending", Reason="", readiness=false. Elapsed: 111.047436ms
Sep 29 15:16:43.888: INFO: Pod "pod-subpath-test-dynamicpv-dg58": Phase="Pending", Reason="", readiness=false. Elapsed: 2.222436737s
Sep 29 15:16:46.000: INFO: Pod "pod-subpath-test-dynamicpv-dg58": Phase="Pending", Reason="", readiness=false. Elapsed: 4.334442785s
Sep 29 15:16:48.111: INFO: Pod "pod-subpath-test-dynamicpv-dg58": Phase="Pending", Reason="", readiness=false. Elapsed: 6.445854104s
Sep 29 15:16:50.222: INFO: Pod "pod-subpath-test-dynamicpv-dg58": Phase="Pending", Reason="", readiness=false. Elapsed: 8.556958581s
Sep 29 15:16:52.334: INFO: Pod "pod-subpath-test-dynamicpv-dg58": Phase="Pending", Reason="", readiness=false. Elapsed: 10.668861239s
... skipping 23 lines ...
Sep 29 15:17:43.039: INFO: Pod "pod-subpath-test-dynamicpv-dg58": Phase="Pending", Reason="", readiness=false. Elapsed: 1m1.37319809s
Sep 29 15:17:45.151: INFO: Pod "pod-subpath-test-dynamicpv-dg58": Phase="Pending", Reason="", readiness=false. Elapsed: 1m3.485711212s
Sep 29 15:17:47.264: INFO: Pod "pod-subpath-test-dynamicpv-dg58": Phase="Pending", Reason="", readiness=false. Elapsed: 1m5.5982191s
Sep 29 15:17:49.376: INFO: Pod "pod-subpath-test-dynamicpv-dg58": Phase="Pending", Reason="", readiness=false. Elapsed: 1m7.710549506s
Sep 29 15:17:51.489: INFO: Pod "pod-subpath-test-dynamicpv-dg58": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m9.823517773s
STEP: Saw pod success
Sep 29 15:17:51.489: INFO: Pod "pod-subpath-test-dynamicpv-dg58" satisfied condition "Succeeded or Failed"
Sep 29 15:17:51.621: INFO: Trying to get logs from node k8s-agentpool1-18436497-vmss000001 pod pod-subpath-test-dynamicpv-dg58 container test-container-volume-dynamicpv-dg58: <nil>
STEP: delete the pod
Sep 29 15:17:51.852: INFO: Waiting for pod pod-subpath-test-dynamicpv-dg58 to disappear
Sep 29 15:17:51.963: INFO: Pod pod-subpath-test-dynamicpv-dg58 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-dg58
Sep 29 15:17:51.963: INFO: Deleting pod "pod-subpath-test-dynamicpv-dg58" in namespace "provisioning-5992"
... skipping 29 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support non-existent path
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:194
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support non-existent path","total":13,"completed":3,"skipped":670,"failed":0}

SSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)(allowExpansion)][Feature:Windows] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:19:04.427: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 111 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support two pods which share the same volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:183
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support two pods which share the same volume","total":20,"completed":1,"skipped":184,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral 
  should create read/write inline ephemeral volume
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:166
... skipping 43 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read/write inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:166
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read/write inline ephemeral volume","total":15,"completed":4,"skipped":158,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:20:08.832: INFO: Driver "test.csi.azure.com" does not support volume expansion - skipping
... skipping 265 lines ...
Sep 29 15:19:18.162: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com5x6f9] to have phase Bound
Sep 29 15:19:18.270: INFO: PersistentVolumeClaim test.csi.azure.com5x6f9 found but phase is Pending instead of Bound.
Sep 29 15:19:20.380: INFO: PersistentVolumeClaim test.csi.azure.com5x6f9 found but phase is Pending instead of Bound.
Sep 29 15:19:22.491: INFO: PersistentVolumeClaim test.csi.azure.com5x6f9 found and phase=Bound (4.329677129s)
STEP: Creating pod pod-subpath-test-dynamicpv-f2ps
STEP: Creating a pod to test subpath
Sep 29 15:19:22.824: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-f2ps" in namespace "provisioning-3859" to be "Succeeded or Failed"
Sep 29 15:19:22.938: INFO: Pod "pod-subpath-test-dynamicpv-f2ps": Phase="Pending", Reason="", readiness=false. Elapsed: 113.097883ms
Sep 29 15:19:25.048: INFO: Pod "pod-subpath-test-dynamicpv-f2ps": Phase="Pending", Reason="", readiness=false. Elapsed: 2.223885237s
Sep 29 15:19:27.160: INFO: Pod "pod-subpath-test-dynamicpv-f2ps": Phase="Pending", Reason="", readiness=false. Elapsed: 4.335074387s
Sep 29 15:19:29.287: INFO: Pod "pod-subpath-test-dynamicpv-f2ps": Phase="Pending", Reason="", readiness=false. Elapsed: 6.462739573s
Sep 29 15:19:31.398: INFO: Pod "pod-subpath-test-dynamicpv-f2ps": Phase="Pending", Reason="", readiness=false. Elapsed: 8.573517673s
Sep 29 15:19:33.513: INFO: Pod "pod-subpath-test-dynamicpv-f2ps": Phase="Pending", Reason="", readiness=false. Elapsed: 10.688269336s
Sep 29 15:19:35.624: INFO: Pod "pod-subpath-test-dynamicpv-f2ps": Phase="Pending", Reason="", readiness=false. Elapsed: 12.799209424s
Sep 29 15:19:37.734: INFO: Pod "pod-subpath-test-dynamicpv-f2ps": Phase="Pending", Reason="", readiness=false. Elapsed: 14.909105431s
Sep 29 15:19:39.845: INFO: Pod "pod-subpath-test-dynamicpv-f2ps": Phase="Pending", Reason="", readiness=false. Elapsed: 17.020516347s
Sep 29 15:19:41.956: INFO: Pod "pod-subpath-test-dynamicpv-f2ps": Phase="Pending", Reason="", readiness=false. Elapsed: 19.131552264s
Sep 29 15:19:44.068: INFO: Pod "pod-subpath-test-dynamicpv-f2ps": Phase="Pending", Reason="", readiness=false. Elapsed: 21.243126464s
Sep 29 15:19:46.178: INFO: Pod "pod-subpath-test-dynamicpv-f2ps": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.353682558s
STEP: Saw pod success
Sep 29 15:19:46.178: INFO: Pod "pod-subpath-test-dynamicpv-f2ps" satisfied condition "Succeeded or Failed"
Sep 29 15:19:46.287: INFO: Trying to get logs from node k8s-agentpool1-18436497-vmss000000 pod pod-subpath-test-dynamicpv-f2ps container test-container-volume-dynamicpv-f2ps: <nil>
STEP: delete the pod
Sep 29 15:19:46.561: INFO: Waiting for pod pod-subpath-test-dynamicpv-f2ps to disappear
Sep 29 15:19:46.670: INFO: Pod pod-subpath-test-dynamicpv-f2ps no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-f2ps
Sep 29 15:19:46.671: INFO: Deleting pod "pod-subpath-test-dynamicpv-f2ps" in namespace "provisioning-3859"
... skipping 23 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support existing directory
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:205
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directory","total":20,"completed":2,"skipped":280,"failed":0}

SSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral 
  should create read/write inline ephemeral volume
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:166
... skipping 49 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read/write inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:166
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read/write inline ephemeral volume","total":13,"completed":4,"skipped":741,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy 
  (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
... skipping 108 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents","total":12,"completed":3,"skipped":302,"failed":0}

SSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:22:30.364: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 115 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","total":18,"completed":4,"skipped":596,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:23:20.086: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 191 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents","total":20,"completed":3,"skipped":289,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: CSI Ephemeral-volume (default fs)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:23:33.970: INFO: Driver "test.csi.azure.com" does not support volume type "CSIInlineVolume" - skipping
... skipping 331 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","total":19,"completed":3,"skipped":532,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (ext3)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:23:39.014: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 98 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (filesystem volmode)] volumeMode
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should not mount / map unused volumes in a pod [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:351
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode should not mount / map unused volumes in a pod [LinuxOnly]","total":19,"completed":4,"skipped":691,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)(allowExpansion)][Feature:Windows] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:24:36.344: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 37 lines ...
Sep 29 15:23:21.188: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com6sstc] to have phase Bound
Sep 29 15:23:21.298: INFO: PersistentVolumeClaim test.csi.azure.com6sstc found but phase is Pending instead of Bound.
Sep 29 15:23:23.410: INFO: PersistentVolumeClaim test.csi.azure.com6sstc found but phase is Pending instead of Bound.
Sep 29 15:23:25.522: INFO: PersistentVolumeClaim test.csi.azure.com6sstc found and phase=Bound (4.333436323s)
STEP: Creating pod pod-subpath-test-dynamicpv-nndn
STEP: Creating a pod to test subpath
Sep 29 15:23:25.856: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-nndn" in namespace "provisioning-3027" to be "Succeeded or Failed"
Sep 29 15:23:25.966: INFO: Pod "pod-subpath-test-dynamicpv-nndn": Phase="Pending", Reason="", readiness=false. Elapsed: 110.503203ms
Sep 29 15:23:28.078: INFO: Pod "pod-subpath-test-dynamicpv-nndn": Phase="Pending", Reason="", readiness=false. Elapsed: 2.222408496s
Sep 29 15:23:30.192: INFO: Pod "pod-subpath-test-dynamicpv-nndn": Phase="Pending", Reason="", readiness=false. Elapsed: 4.335911285s
Sep 29 15:23:32.304: INFO: Pod "pod-subpath-test-dynamicpv-nndn": Phase="Pending", Reason="", readiness=false. Elapsed: 6.448613063s
Sep 29 15:23:34.415: INFO: Pod "pod-subpath-test-dynamicpv-nndn": Phase="Pending", Reason="", readiness=false. Elapsed: 8.559105502s
Sep 29 15:23:36.526: INFO: Pod "pod-subpath-test-dynamicpv-nndn": Phase="Pending", Reason="", readiness=false. Elapsed: 10.670380336s
Sep 29 15:23:38.639: INFO: Pod "pod-subpath-test-dynamicpv-nndn": Phase="Pending", Reason="", readiness=false. Elapsed: 12.783461234s
Sep 29 15:23:40.751: INFO: Pod "pod-subpath-test-dynamicpv-nndn": Phase="Pending", Reason="", readiness=false. Elapsed: 14.895363118s
Sep 29 15:23:42.862: INFO: Pod "pod-subpath-test-dynamicpv-nndn": Phase="Pending", Reason="", readiness=false. Elapsed: 17.006150917s
Sep 29 15:23:44.973: INFO: Pod "pod-subpath-test-dynamicpv-nndn": Phase="Pending", Reason="", readiness=false. Elapsed: 19.116911823s
Sep 29 15:23:47.083: INFO: Pod "pod-subpath-test-dynamicpv-nndn": Phase="Pending", Reason="", readiness=false. Elapsed: 21.227065251s
Sep 29 15:23:49.194: INFO: Pod "pod-subpath-test-dynamicpv-nndn": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.338006684s
STEP: Saw pod success
Sep 29 15:23:49.194: INFO: Pod "pod-subpath-test-dynamicpv-nndn" satisfied condition "Succeeded or Failed"
Sep 29 15:23:49.304: INFO: Trying to get logs from node k8s-agentpool1-18436497-vmss000000 pod pod-subpath-test-dynamicpv-nndn container test-container-subpath-dynamicpv-nndn: <nil>
STEP: delete the pod
Sep 29 15:23:49.580: INFO: Waiting for pod pod-subpath-test-dynamicpv-nndn to disappear
Sep 29 15:23:49.689: INFO: Pod pod-subpath-test-dynamicpv-nndn no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-nndn
Sep 29 15:23:49.689: INFO: Deleting pod "pod-subpath-test-dynamicpv-nndn" in namespace "provisioning-3027"
... skipping 29 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support readOnly file specified in the volumeMount [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:380
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly file specified in the volumeMount [LinuxOnly]","total":18,"completed":5,"skipped":895,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
Sep 29 15:25:02.328: INFO: Running AfterSuite actions on all nodes
Sep 29 15:25:02.328: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func17.2
Sep 29 15:25:02.328: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func8.2
... skipping 53 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read-only inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read-only inline ephemeral volume","total":19,"completed":5,"skipped":859,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (ext4)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:26:12.473: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 158 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:201
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]","total":20,"completed":4,"skipped":732,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
Sep 29 15:26:50.896: INFO: Running AfterSuite actions on all nodes
Sep 29 15:26:50.896: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func17.2
Sep 29 15:26:50.896: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func8.2
... skipping 85 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support volume limits [Serial]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumelimits.go:123
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits should support volume limits [Serial]","total":8,"completed":3,"skipped":756,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:27:48.653: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 82 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support multiple inline ephemeral volumes
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:221
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support multiple inline ephemeral volumes","total":12,"completed":4,"skipped":317,"failed":0}

SSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 29 15:30:36.231: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 174 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support two pods which share the same volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:183
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which share the same volume","total":13,"completed":5,"skipped":823,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
Sep 29 15:35:19.287: INFO: Running AfterSuite actions on all nodes
Sep 29 15:35:19.287: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func17.2
Sep 29 15:35:19.287: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func8.2
... skipping 13 lines ...
Sep 29 15:20:11.467: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage/vsphere.glob..func4.2
Sep 29 15:20:11.467: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage/vsphere.glob..func1.3
Sep 29 15:35:19.342: INFO: Running AfterSuite actions on node 1


Ran 30 of 6664 Specs in 1312.719 seconds
SUCCESS! -- 30 Passed | 0 Failed | 0 Pending | 6634 Skipped 

Ginkgo ran 1 suite in 21m57.946969735s
Test Suite Passed
+ print_logs
+ sed -i s/disk.csi.azure.com/test.csi.azure.com/g deploy/example/storageclass-azuredisk-csi.yaml
+ bash ./hack/verify-examples.sh linux azurepubliccloud ephemeral test
... skipping 76 lines ...
Platform: linux/amd64
Topology Key: topology.test.csi.azure.com/zone

Streaming logs below:
I0929 15:13:18.041464       1 azuredisk.go:136] driver userAgent: test.csi.azure.com/v1.8.0-48d7bd18383c09b02fba2b10739a91587fbfde8c gc/go1.17.1 (amd64-linux) e2e-test
I0929 15:13:18.042773       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W0929 15:13:18.135961       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I0929 15:13:18.135992       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I0929 15:13:18.136002       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I0929 15:13:18.136033       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I0929 15:13:18.136815       1 azure_auth.go:234] Using AzurePublicCloud environment
I0929 15:13:18.136837       1 azure_auth.go:96] azure: using managed identity extension to retrieve access token
I0929 15:13:18.136842       1 azure_auth.go:102] azure: using User Assigned MSI ID to retrieve access token
... skipping 101 lines ...
Platform: linux/amd64
Topology Key: topology.test.csi.azure.com/zone

Streaming logs below:
I0929 15:13:16.678060       1 azuredisk.go:136] driver userAgent: test.csi.azure.com/v1.8.0-48d7bd18383c09b02fba2b10739a91587fbfde8c gc/go1.17.1 (amd64-linux) e2e-test
I0929 15:13:16.679348       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W0929 15:13:16.702773       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I0929 15:13:16.702804       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I0929 15:13:16.702813       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I0929 15:13:16.702842       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I0929 15:13:16.703526       1 azure_auth.go:234] Using AzurePublicCloud environment
I0929 15:13:16.703552       1 azure_auth.go:96] azure: using managed identity extension to retrieve access token
I0929 15:13:16.703557       1 azure_auth.go:102] azure: using User Assigned MSI ID to retrieve access token
... skipping 89 lines ...
I0929 15:13:29.980930       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:13:29.980973       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674","parameters":{"csi.storage.k8s.io/pv/name":"pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674","csi.storage.k8s.io/pvc/name":"test.csi.azure.com2xv9k","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-6988"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:13:30.049374       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:13:30.049409       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a","parameters":{"csi.storage.k8s.io/pv/name":"pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a","csi.storage.k8s.io/pvc/name":"test.csi.azure.comcftb4","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-6425"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:13:30.183276       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:13:30.183303       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5","parameters":{"csi.storage.k8s.io/pv/name":"pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5","csi.storage.k8s.io/pvc/name":"test.csi.azure.com7hrtl","csi.storage.k8s.io/pvc/namespace":"provisioning-9408"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:13:30.272180       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:13:30.272324       1 controllerserver.go:269] begin to create azure disk(pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:13:30.272372       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a StorageAccountType:StandardSSD_LRS Size:5
I0929 15:13:30.372191       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:13:30.372308       1 controllerserver.go:269] begin to create azure disk(pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:13:30.372356       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:13:30.472933       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:13:30.473016       1 controllerserver.go:269] begin to create azure disk(pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:13:30.473034       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:13:30.750815       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:13:30.750843       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-047336da-8095-487e-82fa-794474c45091","parameters":{"csi.storage.k8s.io/pv/name":"pvc-047336da-8095-487e-82fa-794474c45091","csi.storage.k8s.io/pvc/name":"test.csi.azure.comcnd7c","csi.storage.k8s.io/pvc/namespace":"volume-5559"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":7}}]}
I0929 15:13:30.770458       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:13:30.770526       1 controllerserver.go:269] begin to create azure disk(pvc-047336da-8095-487e-82fa-794474c45091) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:13:30.770542       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-047336da-8095-487e-82fa-794474c45091 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:13:31.228730       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:13:31.228758       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-97ce3789-42f8-4295-9c98-534364946c6d","parameters":{"csi.storage.k8s.io/pv/name":"pvc-97ce3789-42f8-4295-9c98-534364946c6d","csi.storage.k8s.io/pvc/name":"test.csi.azure.comlmjhv","csi.storage.k8s.io/pvc/namespace":"provisioning-8114"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:13:31.259578       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-97ce3789-42f8-4295-9c98-534364946c6d, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-97ce3789-42f8-4295-9c98-534364946c6d' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:13:31.259649       1 controllerserver.go:269] begin to create azure disk(pvc-97ce3789-42f8-4295-9c98-534364946c6d) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:13:31.259682       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-97ce3789-42f8-4295-9c98-534364946c6d StorageAccountType:StandardSSD_LRS Size:5
I0929 15:13:32.759972       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a StorageAccountType:StandardSSD_LRS Size:5
I0929 15:13:32.760020       1 controllerserver.go:346] create azure disk(pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a kubernetes.io-created-for-pvc-name:test.csi.azure.comcftb4 kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-6425]) successfully
I0929 15:13:32.760048       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a","csi.storage.k8s.io/pvc/name":"test.csi.azure.comcftb4","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-6425","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a"}}
I0929 15:13:32.790088       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:13:32.790114       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-f0de7246-efb8-4a10-9d79-316f4db64e33","parameters":{"csi.storage.k8s.io/pv/name":"pvc-f0de7246-efb8-4a10-9d79-316f4db64e33","csi.storage.k8s.io/pvc/name":"test.csi.azure.com52wk5","csi.storage.k8s.io/pvc/namespace":"volumemode-8956"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":7}}]}
I0929 15:13:32.810087       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-f0de7246-efb8-4a10-9d79-316f4db64e33, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-f0de7246-efb8-4a10-9d79-316f4db64e33' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:13:32.810150       1 controllerserver.go:269] begin to create azure disk(pvc-f0de7246-efb8-4a10-9d79-316f4db64e33) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:13:32.810166       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-f0de7246-efb8-4a10-9d79-316f4db64e33 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:13:32.816188       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:13:32.816226       1 controllerserver.go:346] create azure disk(pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674 kubernetes.io-created-for-pvc-name:test.csi.azure.com2xv9k kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-6988]) successfully
I0929 15:13:32.816269       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674","csi.storage.k8s.io/pvc/name":"test.csi.azure.com2xv9k","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-6988","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674"}}
I0929 15:13:32.961301       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5 StorageAccountType:StandardSSD_LRS Size:5
... skipping 8 lines ...
I0929 15:13:34.692999       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:13:34.693028       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-fc414f18-aacc-4672-bb32-276cf8803838","parameters":{"csi.storage.k8s.io/pv/name":"pvc-fc414f18-aacc-4672-bb32-276cf8803838","csi.storage.k8s.io/pvc/name":"inline-volume-tester-lf8g6-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-5101"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:13:34.759299       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:13:34.759417       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:13:34.759330       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a","csi.storage.k8s.io/pvc/name":"test.csi.azure.comcftb4","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-6425","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a"}
I0929 15:13:34.759431       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674","csi.storage.k8s.io/pvc/name":"test.csi.azure.com2xv9k","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-6988","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674"}
I0929 15:13:34.761240       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fc414f18-aacc-4672-bb32-276cf8803838, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-fc414f18-aacc-4672-bb32-276cf8803838' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:13:34.761311       1 controllerserver.go:269] begin to create azure disk(pvc-fc414f18-aacc-4672-bb32-276cf8803838) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:13:34.761331       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-fc414f18-aacc-4672-bb32-276cf8803838 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:13:34.781881       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674" to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:13:34.781946       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:13:34.785793       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:13:34.785821       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 17 lines ...
I0929 15:13:35.137900       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-f0de7246-efb8-4a10-9d79-316f4db64e33","csi.storage.k8s.io/pvc/name":"test.csi.azure.com52wk5","csi.storage.k8s.io/pvc/namespace":"volumemode-8956","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-f0de7246-efb8-4a10-9d79-316f4db64e33"}}
I0929 15:13:35.437705       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:13:35.437733       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-047336da-8095-487e-82fa-794474c45091","csi.storage.k8s.io/pvc/name":"test.csi.azure.comcnd7c","csi.storage.k8s.io/pvc/namespace":"volume-5559","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091"}
I0929 15:13:35.460474       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:13:35.460525       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:13:35.460582       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:13:35.477509       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:13:35.860653       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:13:35.860835       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:13:35.860861       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:13:35.944569       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5" lun 1 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-047336da-8095-487e-82fa-794474c45091:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-047336da-8095-487e-82fa-794474c45091  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5  true false 1})]
I0929 15:13:35.944643       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:13:35.944692       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-047336da-8095-487e-82fa-794474c45091:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-047336da-8095-487e-82fa-794474c45091  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5  true false 1})])
I0929 15:13:35.948522       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 2 lines ...
I0929 15:13:35.972319       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:13:35.972375       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-97ce3789-42f8-4295-9c98-534364946c6d" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:13:35.972396       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:13:35.972408       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-97ce3789-42f8-4295-9c98-534364946c6d" lun 1 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-97ce3789-42f8-4295-9c98-534364946c6d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-97ce3789-42f8-4295-9c98-534364946c6d  true false 1})]
I0929 15:13:35.972438       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:13:35.972451       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-97ce3789-42f8-4295-9c98-534364946c6d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-97ce3789-42f8-4295-9c98-534364946c6d  true false 1})])
I0929 15:13:36.113036       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-047336da-8095-487e-82fa-794474c45091:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-047336da-8095-487e-82fa-794474c45091  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:13:36.113214       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:13:36.113231       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:13:36.120486       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-97ce3789-42f8-4295-9c98-534364946c6d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-97ce3789-42f8-4295-9c98-534364946c6d  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:13:36.170774       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091" lun 2 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[]
I0929 15:13:36.170816       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091" attached to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:13:36.170828       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091" to node "k8s-agentpool1-18436497-vmss000001" successfully
I0929 15:13:36.170853       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"2"}}
I0929 15:13:36.273992       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:13:36.274032       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-047336da-8095-487e-82fa-794474c45091","csi.storage.k8s.io/pvc/name":"test.csi.azure.comcnd7c","csi.storage.k8s.io/pvc/namespace":"volume-5559","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091"}
... skipping 21 lines ...
I0929 15:13:37.803153       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000000, refreshing the cache
I0929 15:13:37.846185       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fc414f18-aacc-4672-bb32-276cf8803838" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:13:37.846223       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:13:37.846243       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fc414f18-aacc-4672-bb32-276cf8803838" lun 2 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fc414f18-aacc-4672-bb32-276cf8803838:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc414f18-aacc-4672-bb32-276cf8803838  true false 2})]
I0929 15:13:37.846260       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:13:37.846274       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fc414f18-aacc-4672-bb32-276cf8803838:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc414f18-aacc-4672-bb32-276cf8803838  true false 2})])
I0929 15:13:37.982351       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-f0de7246-efb8-4a10-9d79-316f4db64e33:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f0de7246-efb8-4a10-9d79-316f4db64e33  true false 3})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:13:38.021671       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fc414f18-aacc-4672-bb32-276cf8803838:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc414f18-aacc-4672-bb32-276cf8803838  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:13:45.986340       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a" attached to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:13:45.986377       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a" to node "k8s-agentpool1-18436497-vmss000001" successfully
I0929 15:13:45.986413       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I0929 15:13:46.211981       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5" attached to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:13:46.212017       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5" to node "k8s-agentpool1-18436497-vmss000001" successfully
I0929 15:13:46.212055       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
... skipping 24 lines ...
I0929 15:13:54.904858       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:13:54.904877       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-f0de7246-efb8-4a10-9d79-316f4db64e33" from node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-f0de7246-efb8-4a10-9d79-316f4db64e33:pvc-f0de7246-efb8-4a10-9d79-316f4db64e33]
I0929 15:13:54.904914       1 azure_controller_vmss.go:174] azureDisk - detach disk: name "pvc-f0de7246-efb8-4a10-9d79-316f4db64e33" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-f0de7246-efb8-4a10-9d79-316f4db64e33"
I0929 15:13:54.904947       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-f0de7246-efb8-4a10-9d79-316f4db64e33:pvc-f0de7246-efb8-4a10-9d79-316f4db64e33])
I0929 15:13:58.702123       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:13:58.702152       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0","parameters":{"csi.storage.k8s.io/pv/name":"pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-x6wzv-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-5101"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:13:58.721133       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:13:58.721245       1 controllerserver.go:269] begin to create azure disk(pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:13:58.721261       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:14:01.053643       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:14:01.053714       1 controllerserver.go:346] create azure disk(pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0 kubernetes.io-created-for-pvc-name:inline-volume-tester2-x6wzv-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-5101]) successfully
I0929 15:14:01.053752       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-x6wzv-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-5101","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0"}}
I0929 15:14:01.594210       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
... skipping 51 lines ...
I0929 15:14:37.109483       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674" from node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674:pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674]
I0929 15:14:37.109523       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000000, refreshing the cache
I0929 15:14:37.168886       1 azure_controller_vmss.go:174] azureDisk - detach disk: name "pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674"
I0929 15:14:37.168918       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674:pvc-c0eb4b80-7c23-4fb1-8591-f1c2129a3674])
I0929 15:14:38.044684       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:14:38.044715       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69","parameters":{"csi.storage.k8s.io/pv/name":"pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69","csi.storage.k8s.io/pvc/name":"test.csi.azure.com5lq8g","csi.storage.k8s.io/pvc/namespace":"provisioning-6052"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:14:38.066991       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:14:38.067066       1 controllerserver.go:269] begin to create azure disk(pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:14:38.067080       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:14:40.366459       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:14:40.366499       1 controllerserver.go:346] create azure disk(pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69 kubernetes.io-created-for-pvc-name:test.csi.azure.com5lq8g kubernetes.io-created-for-pvc-namespace:provisioning-6052]) successfully
I0929 15:14:40.366524       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69","csi.storage.k8s.io/pvc/name":"test.csi.azure.com5lq8g","csi.storage.k8s.io/pvc/namespace":"provisioning-6052","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69"}}
I0929 15:14:40.576275       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5:pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5 /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a:pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a]) returned with <nil>
... skipping 26 lines ...
I0929 15:14:55.910094       1 utils.go:102] GRPC response: {}
I0929 15:14:55.910221       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:14:55.910240       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:14:55.998742       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0" lun 0 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a  true false 1})]
I0929 15:14:55.998781       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:14:55.998839       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a  true false 1})])
I0929 15:14:56.172003       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:14:56.172322       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091" from node "k8s-agentpool1-18436497-vmss000001", diskMap: map[]
I0929 15:14:56.172399       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:14:56.172435       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:14:56.230435       1 azure_controller_common.go:369] azureDisk - detach disk(pvc-047336da-8095-487e-82fa-794474c45091, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091) succeeded
I0929 15:14:56.230485       1 controllerserver.go:531] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091 from node k8s-agentpool1-18436497-vmss000001 successfully
I0929 15:14:56.230515       1 utils.go:102] GRPC response: {}
... skipping 23 lines ...
I0929 15:14:57.507865       1 utils.go:102] GRPC response: {}
I0929 15:14:57.507923       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:14:57.507967       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000000, refreshing the cache
I0929 15:14:57.574997       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69" lun 0 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69  true false 0})]
I0929 15:14:57.575034       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:14:57.575072       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69  true false 0})])
I0929 15:14:57.769927       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:15:00.253154       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:15:00.253182       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-815b9aac-6285-4c54-970c-45055d7e4bda","parameters":{"csi.storage.k8s.io/pv/name":"pvc-815b9aac-6285-4c54-970c-45055d7e4bda","csi.storage.k8s.io/pvc/name":"inline-volume-tester-4t8bm-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-1599"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:15:00.276662       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-815b9aac-6285-4c54-970c-45055d7e4bda, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-815b9aac-6285-4c54-970c-45055d7e4bda' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:15:00.276754       1 controllerserver.go:269] begin to create azure disk(pvc-815b9aac-6285-4c54-970c-45055d7e4bda) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:15:00.276789       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-815b9aac-6285-4c54-970c-45055d7e4bda StorageAccountType:StandardSSD_LRS Size:5
I0929 15:15:00.396678       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I0929 15:15:00.396705       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5"}
I0929 15:15:00.396787       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5)
I0929 15:15:00.558117       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 12 lines ...
I0929 15:15:03.875521       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-815b9aac-6285-4c54-970c-45055d7e4bda" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:15:05.637731       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5
I0929 15:15:05.637775       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a8c7d299-cd4b-4b54-b428-a273ecb1f3b5) returned with <nil>
I0929 15:15:05.637798       1 utils.go:102] GRPC response: {}
I0929 15:15:06.902735       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:15:06.902762       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393","parameters":{"csi.storage.k8s.io/pv/name":"pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393","csi.storage.k8s.io/pvc/name":"test.csi.azure.comc6b67","csi.storage.k8s.io/pvc/namespace":"volume-4666"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:15:06.924917       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:15:06.925012       1 controllerserver.go:269] begin to create azure disk(pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:15:06.925052       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:15:09.232549       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:15:09.232590       1 controllerserver.go:346] create azure disk(pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393 kubernetes.io-created-for-pvc-name:test.csi.azure.comc6b67 kubernetes.io-created-for-pvc-namespace:volume-4666]) successfully
I0929 15:15:09.232614       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393","csi.storage.k8s.io/pvc/name":"test.csi.azure.comc6b67","csi.storage.k8s.io/pvc/namespace":"volume-4666","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393"}}
I0929 15:15:10.976550       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:15:10.976577       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1","parameters":{"csi.storage.k8s.io/pv/name":"pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1","csi.storage.k8s.io/pvc/name":"test.csi.azure.coml5ssg","csi.storage.k8s.io/pvc/namespace":"provisioning-9254"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:15:10.995579       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:15:10.995641       1 controllerserver.go:269] begin to create azure disk(pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:15:10.995656       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:15:11.290304       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0" attached to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:15:11.290342       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dff6d43f-4b87-4353-9bec-d4cfe3e9ace0" to node "k8s-agentpool1-18436497-vmss000001" successfully
I0929 15:15:11.290362       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I0929 15:15:11.491944       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-047336da-8095-487e-82fa-794474c45091:pvc-047336da-8095-487e-82fa-794474c45091]) returned with <nil>
... skipping 11 lines ...
I0929 15:15:11.668992       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:15:11.669030       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:15:11.669045       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:15:11.669056       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393" lun 1 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393  true false 1})]
I0929 15:15:11.669114       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:15:11.669130       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393  true false 1})])
I0929 15:15:11.725963       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-815b9aac-6285-4c54-970c-45055d7e4bda:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-815b9aac-6285-4c54-970c-45055d7e4bda  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:15:11.816018       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:15:12.912182       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69" attached to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:15:12.912234       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69" to node "k8s-agentpool1-18436497-vmss000000" successfully
I0929 15:15:12.912254       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I0929 15:15:13.528135       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:15:13.528178       1 controllerserver.go:346] create azure disk(pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1 kubernetes.io-created-for-pvc-name:test.csi.azure.coml5ssg kubernetes.io-created-for-pvc-namespace:provisioning-9254]) successfully
I0929 15:15:13.528202       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1","csi.storage.k8s.io/pvc/name":"test.csi.azure.coml5ssg","csi.storage.k8s.io/pvc/namespace":"provisioning-9254","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1"}}
... skipping 4 lines ...
I0929 15:15:15.693824       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:15:15.774620       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:15:15.774684       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:15:15.774705       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1" lun 3 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1  true false 3})]
I0929 15:15:15.774724       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:15:15.774741       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1  true false 3})])
I0929 15:15:16.061094       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1  true false 3})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:15:17.253960       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I0929 15:15:17.253986       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091"}
I0929 15:15:17.254066       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091)
I0929 15:15:17.604659       1 azure_vmss.go:319] Couldn't find VMSS VM with scaleSetName "k8s-agentpool1-18436497-vmss" and instanceID "1", refreshing the cache
I0929 15:15:22.030954       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I0929 15:15:22.030981       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a"}
... skipping 5 lines ...
I0929 15:15:22.031208       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a:pvc-f63b5642-d0ad-4242-beaf-7a3aa6a5780a])
I0929 15:15:22.449318       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091
I0929 15:15:22.449351       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-047336da-8095-487e-82fa-794474c45091) returned with <nil>
I0929 15:15:22.449369       1 utils.go:102] GRPC response: {}
I0929 15:15:27.471967       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:15:27.472014       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-514aa0d2-2947-41c5-b41f-a440054ca28c","parameters":{"csi.storage.k8s.io/pv/name":"pvc-514aa0d2-2947-41c5-b41f-a440054ca28c","csi.storage.k8s.io/pvc/name":"pvc-q8t4l","csi.storage.k8s.io/pvc/namespace":"provisioning-4139"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":7}}]}
I0929 15:15:27.493173       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-514aa0d2-2947-41c5-b41f-a440054ca28c, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-514aa0d2-2947-41c5-b41f-a440054ca28c' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:15:27.493261       1 controllerserver.go:269] begin to create azure disk(pvc-514aa0d2-2947-41c5-b41f-a440054ca28c) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:15:27.493292       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-514aa0d2-2947-41c5-b41f-a440054ca28c StorageAccountType:StandardSSD_LRS Size:5
I0929 15:15:29.855918       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-514aa0d2-2947-41c5-b41f-a440054ca28c StorageAccountType:StandardSSD_LRS Size:5
I0929 15:15:29.855960       1 controllerserver.go:346] create azure disk(pvc-514aa0d2-2947-41c5-b41f-a440054ca28c) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-514aa0d2-2947-41c5-b41f-a440054ca28c kubernetes.io-created-for-pvc-name:pvc-q8t4l kubernetes.io-created-for-pvc-namespace:provisioning-4139]) successfully
I0929 15:15:29.855986       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-514aa0d2-2947-41c5-b41f-a440054ca28c","csi.storage.k8s.io/pvc/name":"pvc-q8t4l","csi.storage.k8s.io/pvc/namespace":"provisioning-4139","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-514aa0d2-2947-41c5-b41f-a440054ca28c"}}
I0929 15:15:32.855116       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 2 lines ...
I0929 15:15:32.878002       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:15:32.878031       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-514aa0d2-2947-41c5-b41f-a440054ca28c" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:15:32.878049       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:15:32.878063       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-514aa0d2-2947-41c5-b41f-a440054ca28c" lun 3 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-514aa0d2-2947-41c5-b41f-a440054ca28c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-514aa0d2-2947-41c5-b41f-a440054ca28c  true false 3})]
I0929 15:15:32.878081       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:15:32.878100       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-514aa0d2-2947-41c5-b41f-a440054ca28c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-514aa0d2-2947-41c5-b41f-a440054ca28c  true false 3})])
I0929 15:15:33.144130       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-514aa0d2-2947-41c5-b41f-a440054ca28c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-514aa0d2-2947-41c5-b41f-a440054ca28c  true false 3})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:15:36.219813       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1" attached to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:15:36.219850       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1" to node "k8s-agentpool1-18436497-vmss000001" successfully
I0929 15:15:36.219870       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"3"}}
I0929 15:15:36.230552       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:15:36.230574       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1","csi.storage.k8s.io/pvc/name":"test.csi.azure.coml5ssg","csi.storage.k8s.io/pvc/namespace":"provisioning-9254","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1"}
I0929 15:15:36.253115       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1" to node "k8s-agentpool1-18436497-vmss000001".
... skipping 44 lines ...
I0929 15:15:52.166610       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000000","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393"}
I0929 15:15:52.166726       1 controllerserver.go:522] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393 from node k8s-agentpool1-18436497-vmss000000
I0929 15:15:52.166764       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:15:52.166820       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:15:56.686494       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:15:56.686521       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b","parameters":{"csi.storage.k8s.io/pv/name":"pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b","csi.storage.k8s.io/pvc/name":"test.csi.azure.comddz2h","csi.storage.k8s.io/pvc/namespace":"volume-7066"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:15:56.707230       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:15:56.707299       1 controllerserver.go:269] begin to create azure disk(pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:15:56.707312       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b StorageAccountType:StandardSSD_LRS Size:5
I0929 15:15:57.444090       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69:pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69]) returned with <nil>
I0929 15:15:57.444154       1 azure_controller_common.go:369] azureDisk - detach disk(pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69) succeeded
I0929 15:15:57.444193       1 controllerserver.go:531] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69 from node k8s-agentpool1-18436497-vmss000000 successfully
I0929 15:15:57.444205       1 utils.go:102] GRPC response: {}
... skipping 18 lines ...
I0929 15:16:01.494111       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:01.494134       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:16:01.494165       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:01.494176       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b" lun 1 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b  true false 1})]
I0929 15:16:01.494233       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:01.494250       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b  true false 1})])
I0929 15:16:01.696005       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:16:02.291020       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I0929 15:16:02.291053       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1"}
I0929 15:16:02.291161       1 controllerserver.go:522] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1 from node k8s-agentpool1-18436497-vmss000001
I0929 15:16:02.291180       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:02.291202       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:16:02.334062       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 37 lines ...
I0929 15:16:19.219299       1 utils.go:102] GRPC response: {"snapshot":{"creation_time":{"nanos":21907800,"seconds":1632928577},"ready_to_use":true,"size_bytes":5368709120,"snapshot_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/snapshots/snapshot-5bede02b-00f6-4882-bb1d-5f82f72a826c","source_volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-514aa0d2-2947-41c5-b41f-a440054ca28c"}}
I0929 15:16:21.387145       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I0929 15:16:21.387172       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393"}
I0929 15:16:21.387250       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393)
I0929 15:16:21.581527       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:16:21.581590       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-45086e01-6a9e-4d2d-9fe7-3c93ae43cc3b","parameters":{"csi.storage.k8s.io/pv/name":"pvc-45086e01-6a9e-4d2d-9fe7-3c93ae43cc3b","csi.storage.k8s.io/pvc/name":"pvc-bclbw","csi.storage.k8s.io/pvc/namespace":"provisioning-4139"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":7}}],"volume_content_source":{"Type":{"Snapshot":{"snapshot_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/snapshots/snapshot-5bede02b-00f6-4882-bb1d-5f82f72a826c"}}}}
I0929 15:16:21.601046       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-45086e01-6a9e-4d2d-9fe7-3c93ae43cc3b, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-45086e01-6a9e-4d2d-9fe7-3c93ae43cc3b' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:16:21.601140       1 controllerserver.go:269] begin to create azure disk(pvc-45086e01-6a9e-4d2d-9fe7-3c93ae43cc3b) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:16:21.601172       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-45086e01-6a9e-4d2d-9fe7-3c93ae43cc3b StorageAccountType:StandardSSD_LRS Size:5
I0929 15:16:22.362454       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I0929 15:16:22.362480       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000000","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-514aa0d2-2947-41c5-b41f-a440054ca28c"}
I0929 15:16:22.362592       1 controllerserver.go:522] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-514aa0d2-2947-41c5-b41f-a440054ca28c from node k8s-agentpool1-18436497-vmss000000
I0929 15:16:22.362609       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:22.362649       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:23.046215       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:16:23.046271       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d","csi.storage.k8s.io/pvc/name":"test.csi.azure.comfl5gx","csi.storage.k8s.io/pvc/namespace":"provisioning-9857"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:16:23.067800       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:16:23.067874       1 controllerserver.go:269] begin to create azure disk(pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:16:23.067891       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d StorageAccountType:StandardSSD_LRS Size:5
I0929 15:16:23.104411       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69:pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69]) returned with <nil>
I0929 15:16:23.104483       1 azure_controller_common.go:369] azureDisk - detach disk(pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69) succeeded
I0929 15:16:23.104511       1 controllerserver.go:531] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8858c8bd-f9cb-407f-86f9-d6d855bffa69 from node k8s-agentpool1-18436497-vmss000000 successfully
I0929 15:16:23.104524       1 utils.go:102] GRPC response: {}
... skipping 25 lines ...
I0929 15:16:27.870727       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:16:30.876736       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1
I0929 15:16:30.876773       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-466abe3f-fdc5-44a9-87f8-2c9fab50caa1) returned with <nil>
I0929 15:16:30.876789       1 utils.go:102] GRPC response: {}
I0929 15:16:32.747110       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:16:32.747137       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0","parameters":{"csi.storage.k8s.io/pv/name":"pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0","csi.storage.k8s.io/pvc/name":"test.csi.azure.com8cz2p","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-7061"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:16:32.767505       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:16:32.767577       1 controllerserver.go:269] begin to create azure disk(pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:16:32.767593       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:16:35.325702       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:16:35.325745       1 controllerserver.go:346] create azure disk(pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0 kubernetes.io-created-for-pvc-name:test.csi.azure.com8cz2p kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-7061]) successfully
I0929 15:16:35.325772       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0","csi.storage.k8s.io/pvc/name":"test.csi.azure.com8cz2p","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-7061","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0"}}
I0929 15:16:36.951305       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:16:36.951334       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-3ebd3da4-7557-451d-a54b-abc3c8738896","parameters":{"csi.storage.k8s.io/pv/name":"pvc-3ebd3da4-7557-451d-a54b-abc3c8738896","csi.storage.k8s.io/pvc/name":"test.csi.azure.comxwrc5","csi.storage.k8s.io/pvc/namespace":"provisioning-5992"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:16:36.971348       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3ebd3da4-7557-451d-a54b-abc3c8738896, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-3ebd3da4-7557-451d-a54b-abc3c8738896' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:16:36.971439       1 controllerserver.go:269] begin to create azure disk(pvc-3ebd3da4-7557-451d-a54b-abc3c8738896) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:16:36.971452       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-3ebd3da4-7557-451d-a54b-abc3c8738896 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:16:37.510551       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:16:37.510581       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0","csi.storage.k8s.io/pvc/name":"test.csi.azure.com8cz2p","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-7061","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0"}
I0929 15:16:37.533728       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:16:37.533755       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:37.533819       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:16:37.533834       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:37.533844       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0" lun 3 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0  true false 3})]
I0929 15:16:37.533883       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:37.533899       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0  true false 3})])
I0929 15:16:37.723744       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0  true false 3})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:16:38.382484       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-514aa0d2-2947-41c5-b41f-a440054ca28c:pvc-514aa0d2-2947-41c5-b41f-a440054ca28c /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393:pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393]) returned with <nil>
I0929 15:16:38.382553       1 azure_controller_common.go:369] azureDisk - detach disk(pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393) succeeded
I0929 15:16:38.382575       1 controllerserver.go:531] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d6cfdb8d-f964-4c7b-b584-8cc8b9f2d393 from node k8s-agentpool1-18436497-vmss000000 successfully
I0929 15:16:38.382604       1 utils.go:102] GRPC response: {}
I0929 15:16:38.382702       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-514aa0d2-2947-41c5-b41f-a440054ca28c" from node "k8s-agentpool1-18436497-vmss000000", diskMap: map[]
I0929 15:16:38.382724       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 2 lines ...
I0929 15:16:38.514241       1 controllerserver.go:531] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-514aa0d2-2947-41c5-b41f-a440054ca28c from node k8s-agentpool1-18436497-vmss000000 successfully
I0929 15:16:38.514282       1 utils.go:102] GRPC response: {}
I0929 15:16:38.514376       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:38.514408       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-45086e01-6a9e-4d2d-9fe7-3c93ae43cc3b" lun 0 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-45086e01-6a9e-4d2d-9fe7-3c93ae43cc3b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-45086e01-6a9e-4d2d-9fe7-3c93ae43cc3b  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d  true false 1})]
I0929 15:16:38.514432       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:38.514447       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-45086e01-6a9e-4d2d-9fe7-3c93ae43cc3b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-45086e01-6a9e-4d2d-9fe7-3c93ae43cc3b  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d  true false 1})])
I0929 15:16:38.742359       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-45086e01-6a9e-4d2d-9fe7-3c93ae43cc3b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-45086e01-6a9e-4d2d-9fe7-3c93ae43cc3b  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:16:38.742608       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:38.742659       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000000, refreshing the cache
I0929 15:16:38.787298       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d" lun 1 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[]
I0929 15:16:38.787349       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d" attached to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:16:38.787385       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d" to node "k8s-agentpool1-18436497-vmss000000" successfully
I0929 15:16:38.787400       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
... skipping 6 lines ...
I0929 15:16:41.683513       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:41.683556       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3ebd3da4-7557-451d-a54b-abc3c8738896" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:16:41.683589       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:41.683600       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3ebd3da4-7557-451d-a54b-abc3c8738896" lun 4 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ebd3da4-7557-451d-a54b-abc3c8738896:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3ebd3da4-7557-451d-a54b-abc3c8738896  true false 4})]
I0929 15:16:41.683638       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:41.683673       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ebd3da4-7557-451d-a54b-abc3c8738896:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3ebd3da4-7557-451d-a54b-abc3c8738896  true false 4})])
I0929 15:16:41.884902       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ebd3da4-7557-451d-a54b-abc3c8738896:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3ebd3da4-7557-451d-a54b-abc3c8738896  true false 4})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:16:52.546210       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I0929 15:16:52.546237       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b"}
I0929 15:16:52.546347       1 controllerserver.go:522] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b from node k8s-agentpool1-18436497-vmss000001
I0929 15:16:52.546363       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:16:52.546384       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:16:52.550566       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
... skipping 55 lines ...
I0929 15:17:17.169793       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b)
I0929 15:17:22.367281       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b
I0929 15:17:22.367311       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3cb8079a-1171-477f-a5b0-6ec7297a2f3b) returned with <nil>
I0929 15:17:22.367330       1 utils.go:102] GRPC response: {}
I0929 15:17:29.241703       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:29.241731       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-8453ebe9-d537-41c5-ae4a-f7400a6fa04f","parameters":{"csi.storage.k8s.io/pv/name":"pvc-8453ebe9-d537-41c5-ae4a-f7400a6fa04f","csi.storage.k8s.io/pvc/name":"test.csi.azure.coms4mc2","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:29.263433       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8453ebe9-d537-41c5-ae4a-f7400a6fa04f, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-8453ebe9-d537-41c5-ae4a-f7400a6fa04f' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:29.263509       1 controllerserver.go:269] begin to create azure disk(pvc-8453ebe9-d537-41c5-ae4a-f7400a6fa04f) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:29.263525       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-8453ebe9-d537-41c5-ae4a-f7400a6fa04f StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:31.543524       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-8453ebe9-d537-41c5-ae4a-f7400a6fa04f StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:31.543563       1 controllerserver.go:346] create azure disk(pvc-8453ebe9-d537-41c5-ae4a-f7400a6fa04f) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-8453ebe9-d537-41c5-ae4a-f7400a6fa04f kubernetes.io-created-for-pvc-name:test.csi.azure.coms4mc2 kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:31.543605       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-8453ebe9-d537-41c5-ae4a-f7400a6fa04f","csi.storage.k8s.io/pvc/name":"test.csi.azure.coms4mc2","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8453ebe9-d537-41c5-ae4a-f7400a6fa04f"}}
I0929 15:17:33.178191       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d:pvc-ca92ed7a-1dc3-4ca9-8848-41bbd21b092d]) returned with <nil>
... skipping 17 lines ...
I0929 15:17:33.472640       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:17:33.472667       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-815b9aac-6285-4c54-970c-45055d7e4bda" from node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-815b9aac-6285-4c54-970c-45055d7e4bda:pvc-815b9aac-6285-4c54-970c-45055d7e4bda]
E0929 15:17:33.472710       1 azure_controller_vmss.go:183] detach azure disk on node(k8s-agentpool1-18436497-vmss000001): disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-815b9aac-6285-4c54-970c-45055d7e4bda:pvc-815b9aac-6285-4c54-970c-45055d7e4bda]) not found
I0929 15:17:33.472731       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-815b9aac-6285-4c54-970c-45055d7e4bda:pvc-815b9aac-6285-4c54-970c-45055d7e4bda])
I0929 15:17:33.910179       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:33.910221       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-432c9e62-39b8-44ac-bf04-83e34b649318","parameters":{"csi.storage.k8s.io/pv/name":"pvc-432c9e62-39b8-44ac-bf04-83e34b649318","csi.storage.k8s.io/pvc/name":"pvc-glnv8","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:33.927549       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:33.927631       1 controllerserver.go:269] begin to create azure disk(pvc-432c9e62-39b8-44ac-bf04-83e34b649318) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:33.927650       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-432c9e62-39b8-44ac-bf04-83e34b649318 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:34.019918       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:34.019944       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-4c39d09b-b648-4b06-a903-e8005b8114ad","parameters":{"csi.storage.k8s.io/pv/name":"pvc-4c39d09b-b648-4b06-a903-e8005b8114ad","csi.storage.k8s.io/pvc/name":"pvc-zlk49","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:34.058256       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-4c39d09b-b648-4b06-a903-e8005b8114ad, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-4c39d09b-b648-4b06-a903-e8005b8114ad' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:34.058339       1 controllerserver.go:269] begin to create azure disk(pvc-4c39d09b-b648-4b06-a903-e8005b8114ad) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:34.058357       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-4c39d09b-b648-4b06-a903-e8005b8114ad StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:34.133716       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:34.133744       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4","parameters":{"csi.storage.k8s.io/pv/name":"pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4","csi.storage.k8s.io/pvc/name":"pvc-h99vs","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:34.153537       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:34.153615       1 controllerserver.go:269] begin to create azure disk(pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:34.153631       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:34.241903       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:34.241931       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4","parameters":{"csi.storage.k8s.io/pv/name":"pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4","csi.storage.k8s.io/pvc/name":"pvc-tdkc2","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:34.270528       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:34.270606       1 controllerserver.go:269] begin to create azure disk(pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:34.270624       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:34.351584       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:34.351623       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba","parameters":{"csi.storage.k8s.io/pv/name":"pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba","csi.storage.k8s.io/pvc/name":"pvc-xqfqc","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:34.386506       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:34.386588       1 controllerserver.go:269] begin to create azure disk(pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:34.386605       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:34.464755       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:34.464782       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738","parameters":{"csi.storage.k8s.io/pv/name":"pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738","csi.storage.k8s.io/pvc/name":"pvc-7c2vl","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:34.500825       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:34.500906       1 controllerserver.go:269] begin to create azure disk(pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:34.500925       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:34.575107       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:34.575135       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-ed215498-867b-4073-8a88-6dbf534db251","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ed215498-867b-4073-8a88-6dbf534db251","csi.storage.k8s.io/pvc/name":"pvc-sb5kp","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:34.593739       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:34.593811       1 controllerserver.go:269] begin to create azure disk(pvc-ed215498-867b-4073-8a88-6dbf534db251) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:34.593828       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ed215498-867b-4073-8a88-6dbf534db251 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:34.690712       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:34.690739       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-098c3baa-3153-4025-8102-780f961f253c","parameters":{"csi.storage.k8s.io/pv/name":"pvc-098c3baa-3153-4025-8102-780f961f253c","csi.storage.k8s.io/pvc/name":"pvc-ds984","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:34.733708       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:34.733829       1 controllerserver.go:269] begin to create azure disk(pvc-098c3baa-3153-4025-8102-780f961f253c) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:34.733845       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-098c3baa-3153-4025-8102-780f961f253c StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:34.799900       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:34.799926       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e","csi.storage.k8s.io/pvc/name":"pvc-9d827","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:34.818361       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:34.818439       1 controllerserver.go:269] begin to create azure disk(pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:34.818457       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:34.909745       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:34.909776       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71","parameters":{"csi.storage.k8s.io/pv/name":"pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71","csi.storage.k8s.io/pvc/name":"pvc-p7j2r","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:34.929318       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:34.929388       1 controllerserver.go:269] begin to create azure disk(pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:34.929404       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:35.022113       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:35.022140       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9","parameters":{"csi.storage.k8s.io/pv/name":"pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9","csi.storage.k8s.io/pvc/name":"pvc-srlgt","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:35.040733       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:35.040807       1 controllerserver.go:269] begin to create azure disk(pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:35.040825       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:35.132497       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:35.132523       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92","parameters":{"csi.storage.k8s.io/pv/name":"pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92","csi.storage.k8s.io/pvc/name":"pvc-6gxj6","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:35.154757       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:35.154859       1 controllerserver.go:269] begin to create azure disk(pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:35.154877       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:35.246488       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:35.246518       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b","csi.storage.k8s.io/pvc/name":"pvc-pcfp4","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:35.265605       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:35.265685       1 controllerserver.go:269] begin to create azure disk(pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:35.265715       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:35.353426       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:35.353453       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164","parameters":{"csi.storage.k8s.io/pv/name":"pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164","csi.storage.k8s.io/pvc/name":"pvc-zbvfs","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:35.372234       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:35.372307       1 controllerserver.go:269] begin to create azure disk(pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:35.372323       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:35.464850       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:35.464877       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e","parameters":{"csi.storage.k8s.io/pv/name":"pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e","csi.storage.k8s.io/pvc/name":"pvc-9hjjr","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:35.487368       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:35.487445       1 controllerserver.go:269] begin to create azure disk(pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:35.487461       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:35.576051       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:35.576080       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-947490b0-7614-49a1-a791-0edb87f913dd","parameters":{"csi.storage.k8s.io/pv/name":"pvc-947490b0-7614-49a1-a791-0edb87f913dd","csi.storage.k8s.io/pvc/name":"pvc-4s9t8","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:35.608770       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-947490b0-7614-49a1-a791-0edb87f913dd, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-947490b0-7614-49a1-a791-0edb87f913dd' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:35.608863       1 controllerserver.go:269] begin to create azure disk(pvc-947490b0-7614-49a1-a791-0edb87f913dd) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:35.608881       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-947490b0-7614-49a1-a791-0edb87f913dd StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:35.685749       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:35.685783       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a","parameters":{"csi.storage.k8s.io/pv/name":"pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a","csi.storage.k8s.io/pvc/name":"pvc-899bg","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:35.706397       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:35.706478       1 controllerserver.go:269] begin to create azure disk(pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:35.706505       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:35.797596       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:35.797623       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-01886269-4737-4eda-9608-d73aa87e57ce","parameters":{"csi.storage.k8s.io/pv/name":"pvc-01886269-4737-4eda-9608-d73aa87e57ce","csi.storage.k8s.io/pvc/name":"pvc-p2467","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:35.819119       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:35.819180       1 controllerserver.go:269] begin to create azure disk(pvc-01886269-4737-4eda-9608-d73aa87e57ce) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:35.819196       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-01886269-4737-4eda-9608-d73aa87e57ce StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:35.916537       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:35.916563       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-24916d34-a3fa-440c-911d-d13576ee991b","parameters":{"csi.storage.k8s.io/pv/name":"pvc-24916d34-a3fa-440c-911d-d13576ee991b","csi.storage.k8s.io/pvc/name":"pvc-55ttq","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:35.940383       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-24916d34-a3fa-440c-911d-d13576ee991b, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-24916d34-a3fa-440c-911d-d13576ee991b' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:35.940448       1 controllerserver.go:269] begin to create azure disk(pvc-24916d34-a3fa-440c-911d-d13576ee991b) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:35.940464       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-24916d34-a3fa-440c-911d-d13576ee991b StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:36.018946       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:36.018973       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-3b6aa044-fbdf-4221-a270-96bc83511b59","parameters":{"csi.storage.k8s.io/pv/name":"pvc-3b6aa044-fbdf-4221-a270-96bc83511b59","csi.storage.k8s.io/pvc/name":"pvc-5wbpj","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:36.049780       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3b6aa044-fbdf-4221-a270-96bc83511b59, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-3b6aa044-fbdf-4221-a270-96bc83511b59' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:36.049845       1 controllerserver.go:269] begin to create azure disk(pvc-3b6aa044-fbdf-4221-a270-96bc83511b59) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:36.049860       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-3b6aa044-fbdf-4221-a270-96bc83511b59 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:36.131391       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:36.131426       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4","parameters":{"csi.storage.k8s.io/pv/name":"pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4","csi.storage.k8s.io/pvc/name":"pvc-8s4p8","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:36.150399       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:36.150496       1 controllerserver.go:269] begin to create azure disk(pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:36.150510       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:36.224416       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-432c9e62-39b8-44ac-bf04-83e34b649318 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:36.224487       1 controllerserver.go:346] create azure disk(pvc-432c9e62-39b8-44ac-bf04-83e34b649318) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-432c9e62-39b8-44ac-bf04-83e34b649318 kubernetes.io-created-for-pvc-name:pvc-glnv8 kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:36.224515       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-432c9e62-39b8-44ac-bf04-83e34b649318","csi.storage.k8s.io/pvc/name":"pvc-glnv8","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318"}}
I0929 15:17:36.244545       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:36.244568       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-69b06f70-c974-4347-b57a-f9254747d08c","parameters":{"csi.storage.k8s.io/pv/name":"pvc-69b06f70-c974-4347-b57a-f9254747d08c","csi.storage.k8s.io/pvc/name":"pvc-mgpgp","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:36.264428       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:36.264528       1 controllerserver.go:269] begin to create azure disk(pvc-69b06f70-c974-4347-b57a-f9254747d08c) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:36.264543       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-69b06f70-c974-4347-b57a-f9254747d08c StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:36.361936       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:36.361958       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-12718de8-a30a-4b61-a208-606b8c0021d7","parameters":{"csi.storage.k8s.io/pv/name":"pvc-12718de8-a30a-4b61-a208-606b8c0021d7","csi.storage.k8s.io/pvc/name":"pvc-886zj","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:36.381066       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:36.381181       1 controllerserver.go:269] begin to create azure disk(pvc-12718de8-a30a-4b61-a208-606b8c0021d7) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:36.381195       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-12718de8-a30a-4b61-a208-606b8c0021d7 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:36.466866       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:36.466935       1 controllerserver.go:346] create azure disk(pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4 kubernetes.io-created-for-pvc-name:pvc-h99vs kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:36.466977       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4","csi.storage.k8s.io/pvc/name":"pvc-h99vs","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4"}}
I0929 15:17:36.471482       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:36.471503       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35","parameters":{"csi.storage.k8s.io/pv/name":"pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35","csi.storage.k8s.io/pvc/name":"pvc-g56gk","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:36.490226       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:36.490276       1 controllerserver.go:269] begin to create azure disk(pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:36.490310       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:36.577868       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:36.577957       1 controllerserver.go:346] create azure disk(pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4 kubernetes.io-created-for-pvc-name:pvc-tdkc2 kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:36.577979       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4","csi.storage.k8s.io/pvc/name":"pvc-tdkc2","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4"}}
I0929 15:17:36.580746       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:36.580767       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577","csi.storage.k8s.io/pvc/name":"pvc-zn99l","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:36.600441       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:36.600526       1 controllerserver.go:269] begin to create azure disk(pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:36.600555       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:36.664658       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-4c39d09b-b648-4b06-a903-e8005b8114ad StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:36.664712       1 controllerserver.go:346] create azure disk(pvc-4c39d09b-b648-4b06-a903-e8005b8114ad) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-4c39d09b-b648-4b06-a903-e8005b8114ad kubernetes.io-created-for-pvc-name:pvc-zlk49 kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:36.664736       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-4c39d09b-b648-4b06-a903-e8005b8114ad","csi.storage.k8s.io/pvc/name":"pvc-zlk49","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-4c39d09b-b648-4b06-a903-e8005b8114ad"}}
I0929 15:17:36.739547       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:36.739588       1 controllerserver.go:346] create azure disk(pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba kubernetes.io-created-for-pvc-name:pvc-xqfqc kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:36.739615       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba","csi.storage.k8s.io/pvc/name":"pvc-xqfqc","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba"}}
I0929 15:17:36.758892       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:36.758914       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030","parameters":{"csi.storage.k8s.io/pv/name":"pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030","csi.storage.k8s.io/pvc/name":"pvc-mlz6g","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:36.786594       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:36.786714       1 controllerserver.go:269] begin to create azure disk(pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:36.786746       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:36.813886       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:36.813923       1 controllerserver.go:346] create azure disk(pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738 kubernetes.io-created-for-pvc-name:pvc-7c2vl kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:36.813946       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738","csi.storage.k8s.io/pvc/name":"pvc-7c2vl","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738"}}
I0929 15:17:36.905309       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-ed215498-867b-4073-8a88-6dbf534db251 StorageAccountType:StandardSSD_LRS Size:5
... skipping 4 lines ...
I0929 15:17:37.060892       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-098c3baa-3153-4025-8102-780f961f253c","csi.storage.k8s.io/pvc/name":"pvc-ds984","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c"}}
I0929 15:17:37.141357       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:37.141443       1 controllerserver.go:346] create azure disk(pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e kubernetes.io-created-for-pvc-name:pvc-9d827 kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:37.141485       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e","csi.storage.k8s.io/pvc/name":"pvc-9d827","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e"}}
I0929 15:17:37.158227       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:37.158247       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6","parameters":{"csi.storage.k8s.io/pv/name":"pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6","csi.storage.k8s.io/pvc/name":"pvc-nm8dz","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:37.185088       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:37.185167       1 controllerserver.go:269] begin to create azure disk(pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:37.185181       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:37.208676       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:37.208697       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24","parameters":{"csi.storage.k8s.io/pv/name":"pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24","csi.storage.k8s.io/pvc/name":"pvc-l4ppb","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:37.228167       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:37.228256       1 controllerserver.go:269] begin to create azure disk(pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:37.228267       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:37.259346       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:37.259367       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-5b465967-3761-4776-94d8-06f5b348f6a0","parameters":{"csi.storage.k8s.io/pv/name":"pvc-5b465967-3761-4776-94d8-06f5b348f6a0","csi.storage.k8s.io/pvc/name":"pvc-gqsvm","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:37.269103       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:37.269169       1 controllerserver.go:346] create azure disk(pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71 kubernetes.io-created-for-pvc-name:pvc-p7j2r kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:37.269205       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71","csi.storage.k8s.io/pvc/name":"pvc-p7j2r","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71"}}
I0929 15:17:37.277934       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5b465967-3761-4776-94d8-06f5b348f6a0, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-5b465967-3761-4776-94d8-06f5b348f6a0' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:37.278020       1 controllerserver.go:269] begin to create azure disk(pvc-5b465967-3761-4776-94d8-06f5b348f6a0) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:37.278031       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-5b465967-3761-4776-94d8-06f5b348f6a0 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:37.363862       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:37.363915       1 controllerserver.go:346] create azure disk(pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9 kubernetes.io-created-for-pvc-name:pvc-srlgt kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:37.363971       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9","csi.storage.k8s.io/pvc/name":"pvc-srlgt","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9"}}
I0929 15:17:37.491079       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:37.491137       1 controllerserver.go:346] create azure disk(pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92 kubernetes.io-created-for-pvc-name:pvc-6gxj6 kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:37.491181       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92","csi.storage.k8s.io/pvc/name":"pvc-6gxj6","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92"}}
I0929 15:17:37.571044       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:37.571095       1 controllerserver.go:346] create azure disk(pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b kubernetes.io-created-for-pvc-name:pvc-pcfp4 kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:37.571119       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b","csi.storage.k8s.io/pvc/name":"pvc-pcfp4","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b"}}
I0929 15:17:37.660270       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:37.660309       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc","parameters":{"csi.storage.k8s.io/pv/name":"pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc","csi.storage.k8s.io/pvc/name":"pvc-skmnf","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:37.679073       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:37.679133       1 controllerserver.go:269] begin to create azure disk(pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:37.679148       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:37.725811       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:37.725874       1 controllerserver.go:346] create azure disk(pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164 kubernetes.io-created-for-pvc-name:pvc-zbvfs kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:37.725903       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164","csi.storage.k8s.io/pvc/name":"pvc-zbvfs","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164"}}
I0929 15:17:37.759463       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:37.759485       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e","parameters":{"csi.storage.k8s.io/pv/name":"pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e","csi.storage.k8s.io/pvc/name":"pvc-49xg5","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:37.778190       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:37.778273       1 controllerserver.go:269] begin to create azure disk(pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:37.778286       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:37.860041       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:37.860103       1 controllerserver.go:346] create azure disk(pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e kubernetes.io-created-for-pvc-name:pvc-9hjjr kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:37.860121       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e","csi.storage.k8s.io/pvc/name":"pvc-9hjjr","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e"}}
I0929 15:17:37.944168       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-947490b0-7614-49a1-a791-0edb87f913dd StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:37.944200       1 controllerserver.go:346] create azure disk(pvc-947490b0-7614-49a1-a791-0edb87f913dd) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-947490b0-7614-49a1-a791-0edb87f913dd kubernetes.io-created-for-pvc-name:pvc-4s9t8 kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:37.944219       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-947490b0-7614-49a1-a791-0edb87f913dd","csi.storage.k8s.io/pvc/name":"pvc-4s9t8","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-947490b0-7614-49a1-a791-0edb87f913dd"}}
I0929 15:17:37.982129       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:37.982303       1 controllerserver.go:346] create azure disk(pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a kubernetes.io-created-for-pvc-name:pvc-899bg kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:37.982341       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a","csi.storage.k8s.io/pvc/name":"pvc-899bg","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a"}}
I0929 15:17:38.059834       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:38.059939       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63","parameters":{"csi.storage.k8s.io/pv/name":"pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63","csi.storage.k8s.io/pvc/name":"pvc-cjn8w","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:38.091572       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:38.091690       1 controllerserver.go:269] begin to create azure disk(pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:38.091706       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:38.181862       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-01886269-4737-4eda-9608-d73aa87e57ce StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:38.181904       1 controllerserver.go:346] create azure disk(pvc-01886269-4737-4eda-9608-d73aa87e57ce) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-01886269-4737-4eda-9608-d73aa87e57ce kubernetes.io-created-for-pvc-name:pvc-p2467 kubernetes.io-created-for-pvc-namespace:volumelimits-8887]) successfully
I0929 15:17:38.181931       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-01886269-4737-4eda-9608-d73aa87e57ce","csi.storage.k8s.io/pvc/name":"pvc-p2467","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce"}}
I0929 15:17:38.276817       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-24916d34-a3fa-440c-911d-d13576ee991b StorageAccountType:StandardSSD_LRS Size:5
... skipping 61 lines ...
I0929 15:17:48.440896       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-815b9aac-6285-4c54-970c-45055d7e4bda)
I0929 15:17:53.635443       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-815b9aac-6285-4c54-970c-45055d7e4bda
I0929 15:17:53.635474       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-815b9aac-6285-4c54-970c-45055d7e4bda) returned with <nil>
I0929 15:17:53.635494       1 utils.go:102] GRPC response: {}
I0929 15:17:54.184417       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:54.184444       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb","csi.storage.k8s.io/pvc/name":"test.csi.azure.comr7xgc","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-8602"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:54.214982       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:54.215079       1 controllerserver.go:269] begin to create azure disk(pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:54.215093       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:56.509773       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb StorageAccountType:StandardSSD_LRS Size:5
I0929 15:17:56.509818       1 controllerserver.go:346] create azure disk(pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb kubernetes.io-created-for-pvc-name:test.csi.azure.comr7xgc kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-8602]) successfully
I0929 15:17:56.509842       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb","csi.storage.k8s.io/pvc/name":"test.csi.azure.comr7xgc","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-8602","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb"}}
I0929 15:17:57.815142       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 10 lines ...
I0929 15:17:58.910058       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:17:58.972450       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:17:58.972506       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:17:58.972524       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb" lun 0 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb  true false 0})]
I0929 15:17:58.972541       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:17:58.972555       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb  true false 0})])
I0929 15:17:59.127942       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:17:59.415473       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:17:59.415500       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-5aa0369c-26f9-463b-98bd-ccd821476413","parameters":{"csi.storage.k8s.io/pv/name":"pvc-5aa0369c-26f9-463b-98bd-ccd821476413","csi.storage.k8s.io/pvc/name":"test.csi.azure.comfrs88","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-5421"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:17:59.438385       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:17:59.438504       1 controllerserver.go:269] begin to create azure disk(pvc-5aa0369c-26f9-463b-98bd-ccd821476413) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:17:59.438521       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-5aa0369c-26f9-463b-98bd-ccd821476413 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:18:01.797328       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-5aa0369c-26f9-463b-98bd-ccd821476413 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:18:01.797383       1 controllerserver.go:346] create azure disk(pvc-5aa0369c-26f9-463b-98bd-ccd821476413) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-5aa0369c-26f9-463b-98bd-ccd821476413 kubernetes.io-created-for-pvc-name:test.csi.azure.comfrs88 kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-5421]) successfully
I0929 15:18:01.797408       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-5aa0369c-26f9-463b-98bd-ccd821476413","csi.storage.k8s.io/pvc/name":"test.csi.azure.comfrs88","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-5421","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413"}}
I0929 15:18:02.926975       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
... skipping 52 lines ...
I0929 15:18:26.179075       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-514aa0d2-2947-41c5-b41f-a440054ca28c) returned with <nil>
I0929 15:18:26.179093       1 utils.go:102] GRPC response: {}
I0929 15:18:26.496710       1 controllerserver.go:959] delete snapshot(snapshot-5bede02b-00f6-4882-bb1d-5f82f72a826c) under rg(kubetest-xhnqwtqz) successfully
I0929 15:18:26.496750       1 utils.go:102] GRPC response: {}
I0929 15:18:31.969500       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:18:31.969526       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-ad922f05-173f-453f-8871-e20b206862e3","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ad922f05-173f-453f-8871-e20b206862e3","csi.storage.k8s.io/pvc/name":"inline-volume-tester-9bqtl-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-8071"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:18:32.006235       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ad922f05-173f-453f-8871-e20b206862e3, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ad922f05-173f-453f-8871-e20b206862e3' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:18:32.006320       1 controllerserver.go:269] begin to create azure disk(pvc-ad922f05-173f-453f-8871-e20b206862e3) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:18:32.006357       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ad922f05-173f-453f-8871-e20b206862e3 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:18:34.322820       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-ad922f05-173f-453f-8871-e20b206862e3 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:18:34.322865       1 controllerserver.go:346] create azure disk(pvc-ad922f05-173f-453f-8871-e20b206862e3) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-ad922f05-173f-453f-8871-e20b206862e3 kubernetes.io-created-for-pvc-name:inline-volume-tester-9bqtl-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-8071]) successfully
I0929 15:18:34.322894       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ad922f05-173f-453f-8871-e20b206862e3","csi.storage.k8s.io/pvc/name":"inline-volume-tester-9bqtl-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-8071","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ad922f05-173f-453f-8871-e20b206862e3"}}
I0929 15:18:35.078666       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 12 lines ...
I0929 15:18:38.842816       1 controllerserver.go:522] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0 from node k8s-agentpool1-18436497-vmss000001
I0929 15:18:38.842825       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:18:38.899383       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413" lun 1 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5aa0369c-26f9-463b-98bd-ccd821476413  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ad922f05-173f-453f-8871-e20b206862e3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ad922f05-173f-453f-8871-e20b206862e3  true false 2})]
I0929 15:18:38.899430       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:18:38.899454       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5aa0369c-26f9-463b-98bd-ccd821476413  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ad922f05-173f-453f-8871-e20b206862e3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ad922f05-173f-453f-8871-e20b206862e3  true false 2})])
I0929 15:18:38.899712       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:18:39.071225       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5aa0369c-26f9-463b-98bd-ccd821476413  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ad922f05-173f-453f-8871-e20b206862e3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ad922f05-173f-453f-8871-e20b206862e3  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:18:39.071461       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:18:39.071493       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:18:39.136623       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ad922f05-173f-453f-8871-e20b206862e3" lun 2 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[]
I0929 15:18:39.136666       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ad922f05-173f-453f-8871-e20b206862e3" attached to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:18:39.136678       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ad922f05-173f-453f-8871-e20b206862e3" to node "k8s-agentpool1-18436497-vmss000001" successfully
I0929 15:18:39.136693       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"2"}}
... skipping 6 lines ...
I0929 15:18:39.522168       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:18:39.522225       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:18:39.522259       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:18:39.522271       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0" lun 0 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0  true false 0})]
I0929 15:18:39.522291       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:18:39.522305       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0  true false 0})])
I0929 15:18:39.669758       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:18:43.274710       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I0929 15:18:43.274740       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb"}
I0929 15:18:43.274862       1 controllerserver.go:522] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb from node k8s-agentpool1-18436497-vmss000001
I0929 15:18:43.274875       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:18:43.274920       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:18:43.283398       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
... skipping 10 lines ...
I0929 15:18:55.262269       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3ebd3da4-7557-451d-a54b-abc3c8738896)
I0929 15:19:00.498703       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3ebd3da4-7557-451d-a54b-abc3c8738896
I0929 15:19:00.498728       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3ebd3da4-7557-451d-a54b-abc3c8738896) returned with <nil>
I0929 15:19:00.498769       1 utils.go:102] GRPC response: {}
I0929 15:19:05.920060       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:19:05.920107       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-dee7f437-d435-49ec-97ba-e4d0144153f4","parameters":{"csi.storage.k8s.io/pv/name":"pvc-dee7f437-d435-49ec-97ba-e4d0144153f4","csi.storage.k8s.io/pvc/name":"inline-volume-tester-sm6qn-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-3159"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:19:05.939595       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dee7f437-d435-49ec-97ba-e4d0144153f4, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-dee7f437-d435-49ec-97ba-e4d0144153f4' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:19:05.939678       1 controllerserver.go:269] begin to create azure disk(pvc-dee7f437-d435-49ec-97ba-e4d0144153f4) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:19:05.939695       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-dee7f437-d435-49ec-97ba-e4d0144153f4 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:19:08.324779       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-dee7f437-d435-49ec-97ba-e4d0144153f4 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:19:08.324859       1 controllerserver.go:346] create azure disk(pvc-dee7f437-d435-49ec-97ba-e4d0144153f4) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-dee7f437-d435-49ec-97ba-e4d0144153f4 kubernetes.io-created-for-pvc-name:inline-volume-tester-sm6qn-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-3159]) successfully
I0929 15:19:08.324887       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-dee7f437-d435-49ec-97ba-e4d0144153f4","csi.storage.k8s.io/pvc/name":"inline-volume-tester-sm6qn-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-3159","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dee7f437-d435-49ec-97ba-e4d0144153f4"}}
I0929 15:19:08.636589       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fc414f18-aacc-4672-bb32-276cf8803838:pvc-fc414f18-aacc-4672-bb32-276cf8803838]) returned with <nil>
... skipping 31 lines ...
I0929 15:19:10.784059       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fc414f18-aacc-4672-bb32-276cf8803838)
I0929 15:19:15.967053       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fc414f18-aacc-4672-bb32-276cf8803838
I0929 15:19:15.967101       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fc414f18-aacc-4672-bb32-276cf8803838) returned with <nil>
I0929 15:19:15.967119       1 utils.go:102] GRPC response: {}
I0929 15:19:18.111846       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:19:18.111876       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2","parameters":{"csi.storage.k8s.io/pv/name":"pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2","csi.storage.k8s.io/pvc/name":"test.csi.azure.com5x6f9","csi.storage.k8s.io/pvc/namespace":"provisioning-3859"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:19:18.133212       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:19:18.133288       1 controllerserver.go:269] begin to create azure disk(pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:19:18.133312       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:19:20.593751       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:19:20.593849       1 controllerserver.go:346] create azure disk(pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2 kubernetes.io-created-for-pvc-name:test.csi.azure.com5x6f9 kubernetes.io-created-for-pvc-namespace:provisioning-3859]) successfully
I0929 15:19:20.593890       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2","csi.storage.k8s.io/pvc/name":"test.csi.azure.com5x6f9","csi.storage.k8s.io/pvc/namespace":"provisioning-3859","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2"}}
I0929 15:19:22.909460       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 2 lines ...
I0929 15:19:22.935147       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:19:22.935187       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:19:22.935220       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:19:22.935249       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2" lun 1 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2  true false 1})]
I0929 15:19:22.935273       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:19:22.935305       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2  true false 1})])
I0929 15:19:23.076272       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:19:33.166285       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2" attached to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:19:33.166322       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d5ff3874-d8af-4916-b7da-62c44207d8f2" to node "k8s-agentpool1-18436497-vmss000000" successfully
I0929 15:19:33.166379       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
I0929 15:19:33.495812       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I0929 15:19:33.495839       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ad922f05-173f-453f-8871-e20b206862e3"}
I0929 15:19:33.495946       1 controllerserver.go:522] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ad922f05-173f-453f-8871-e20b206862e3 from node k8s-agentpool1-18436497-vmss000001
... skipping 5 lines ...
I0929 15:19:34.878417       1 utils.go:102] GRPC response: {}
I0929 15:19:34.878479       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:19:34.878505       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:19:34.942269       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dee7f437-d435-49ec-97ba-e4d0144153f4" lun 0 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dee7f437-d435-49ec-97ba-e4d0144153f4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dee7f437-d435-49ec-97ba-e4d0144153f4  true false 0})]
I0929 15:19:34.942314       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:19:34.942339       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dee7f437-d435-49ec-97ba-e4d0144153f4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dee7f437-d435-49ec-97ba-e4d0144153f4  true false 0})])
I0929 15:19:35.094191       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dee7f437-d435-49ec-97ba-e4d0144153f4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dee7f437-d435-49ec-97ba-e4d0144153f4  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:19:35.094405       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ad922f05-173f-453f-8871-e20b206862e3" from node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ad922f05-173f-453f-8871-e20b206862e3:pvc-ad922f05-173f-453f-8871-e20b206862e3]
I0929 15:19:35.094434       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:19:35.156748       1 azure_controller_vmss.go:174] azureDisk - detach disk: name "pvc-ad922f05-173f-453f-8871-e20b206862e3" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ad922f05-173f-453f-8871-e20b206862e3"
I0929 15:19:35.156785       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ad922f05-173f-453f-8871-e20b206862e3:pvc-ad922f05-173f-453f-8871-e20b206862e3])
I0929 15:19:35.831884       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:19:35.831915       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb","csi.storage.k8s.io/pvc/name":"test.csi.azure.comr7xgc","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-8602","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb"}
I0929 15:19:35.856573       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb" to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:19:35.856604       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:19:35.856668       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:19:35.856685       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:19:35.856696       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb" lun 2 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb  true false 2})]
I0929 15:19:35.856728       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:19:35.856765       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb  true false 2})])
I0929 15:19:36.023205       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:19:51.130363       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb" attached to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:19:51.130396       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb" to node "k8s-agentpool1-18436497-vmss000000" successfully
I0929 15:19:51.130430       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"2"}}
I0929 15:19:51.141799       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:19:51.141817       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb","csi.storage.k8s.io/pvc/name":"test.csi.azure.comr7xgc","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-8602","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb"}
I0929 15:19:51.165240       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb" to node "k8s-agentpool1-18436497-vmss000000".
... skipping 54 lines ...
I0929 15:20:23.831658       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:23.831678       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0" from node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0:pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0]
I0929 15:20:23.831707       1 azure_controller_vmss.go:174] azureDisk - detach disk: name "pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0"
I0929 15:20:23.831718       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0:pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0])
I0929 15:20:29.238907       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:20:29.238953       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-76a88138-4681-46a5-998a-b01181972fdb","parameters":{"csi.storage.k8s.io/pv/name":"pvc-76a88138-4681-46a5-998a-b01181972fdb","csi.storage.k8s.io/pvc/name":"test.csi.azure.com89zb6","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-8741"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:20:29.257859       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-76a88138-4681-46a5-998a-b01181972fdb, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-76a88138-4681-46a5-998a-b01181972fdb' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:20:29.257950       1 controllerserver.go:269] begin to create azure disk(pvc-76a88138-4681-46a5-998a-b01181972fdb) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:20:29.257975       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-76a88138-4681-46a5-998a-b01181972fdb StorageAccountType:StandardSSD_LRS Size:5
I0929 15:20:31.594618       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-76a88138-4681-46a5-998a-b01181972fdb StorageAccountType:StandardSSD_LRS Size:5
I0929 15:20:31.594681       1 controllerserver.go:346] create azure disk(pvc-76a88138-4681-46a5-998a-b01181972fdb) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-76a88138-4681-46a5-998a-b01181972fdb kubernetes.io-created-for-pvc-name:test.csi.azure.com89zb6 kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-8741]) successfully
I0929 15:20:31.594705       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-76a88138-4681-46a5-998a-b01181972fdb","csi.storage.k8s.io/pvc/name":"test.csi.azure.com89zb6","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-8741","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-76a88138-4681-46a5-998a-b01181972fdb"}}
I0929 15:20:33.856950       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
... skipping 175 lines ...
I0929 15:20:54.343876       1 utils.go:102] GRPC response: {}
I0929 15:20:54.343970       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:54.343987       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:20:54.430562       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c" lun 30 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-01886269-4737-4eda-9608-d73aa87e57ce  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738  true false 29}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-098c3baa-3153-4025-8102-780f961f253c  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-12718de8-a30a-4b61-a208-606b8c0021d7  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-24916d34-a3fa-440c-911d-d13576ee991b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-24916d34-a3fa-440c-911d-d13576ee991b  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71  true false 13}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3b6aa044-fbdf-4221-a270-96bc83511b59:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3b6aa044-fbdf-4221-a270-96bc83511b59  true false 8}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-432c9e62-39b8-44ac-bf04-83e34b649318  true false 22}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-4c39d09b-b648-4b06-a903-e8005b8114ad:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4c39d09b-b648-4b06-a903-e8005b8114ad  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5b465967-3761-4776-94d8-06f5b348f6a0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5b465967-3761-4776-94d8-06f5b348f6a0  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-69b06f70-c974-4347-b57a-f9254747d08c  true false 4}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6  true false 3}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-947490b0-7614-49a1-a791-0edb87f913dd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-947490b0-7614-49a1-a791-0edb87f913dd  true false 28}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4  true false 5}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4  true false 23}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a  true false 16}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e  true false 26}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577  true false 24}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed215498-867b-4073-8a88-6dbf534db251  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba  true false 27})]
I0929 15:20:54.430644       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:54.430719       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-01886269-4737-4eda-9608-d73aa87e57ce  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738  true false 29}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-098c3baa-3153-4025-8102-780f961f253c  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-12718de8-a30a-4b61-a208-606b8c0021d7  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-24916d34-a3fa-440c-911d-d13576ee991b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-24916d34-a3fa-440c-911d-d13576ee991b  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71  true false 13}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3b6aa044-fbdf-4221-a270-96bc83511b59:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3b6aa044-fbdf-4221-a270-96bc83511b59  true false 8}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-432c9e62-39b8-44ac-bf04-83e34b649318  true false 22}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-4c39d09b-b648-4b06-a903-e8005b8114ad:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4c39d09b-b648-4b06-a903-e8005b8114ad  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5b465967-3761-4776-94d8-06f5b348f6a0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5b465967-3761-4776-94d8-06f5b348f6a0  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-69b06f70-c974-4347-b57a-f9254747d08c  true false 4}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6  true false 3}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-947490b0-7614-49a1-a791-0edb87f913dd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-947490b0-7614-49a1-a791-0edb87f913dd  true false 28}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4  true false 5}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4  true false 23}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a  true false 16}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e  true false 26}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577  true false 24}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed215498-867b-4073-8a88-6dbf534db251  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba  true false 27})])
I0929 15:20:54.496785       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:20:54.496809       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:20:54.496877       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-01886269-4737-4eda-9608-d73aa87e57ce  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738  true false 29}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-098c3baa-3153-4025-8102-780f961f253c  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-12718de8-a30a-4b61-a208-606b8c0021d7  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-24916d34-a3fa-440c-911d-d13576ee991b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-24916d34-a3fa-440c-911d-d13576ee991b  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71  true false 13}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3b6aa044-fbdf-4221-a270-96bc83511b59:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3b6aa044-fbdf-4221-a270-96bc83511b59  true false 8}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-432c9e62-39b8-44ac-bf04-83e34b649318  true false 22}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-4c39d09b-b648-4b06-a903-e8005b8114ad:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4c39d09b-b648-4b06-a903-e8005b8114ad  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5b465967-3761-4776-94d8-06f5b348f6a0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5b465967-3761-4776-94d8-06f5b348f6a0  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-69b06f70-c974-4347-b57a-f9254747d08c  true false 4}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6  true false 3}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-947490b0-7614-49a1-a791-0edb87f913dd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-947490b0-7614-49a1-a791-0edb87f913dd  true false 28}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4  true false 5}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4  true false 23}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a  true false 16}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e  true false 26}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577  true false 24}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed215498-867b-4073-8a88-6dbf534db251  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba  true false 27})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:20:54.497060       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-01886269-4737-4eda-9608-d73aa87e57ce  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738  true false 29}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-098c3baa-3153-4025-8102-780f961f253c  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-12718de8-a30a-4b61-a208-606b8c0021d7  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-24916d34-a3fa-440c-911d-d13576ee991b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-24916d34-a3fa-440c-911d-d13576ee991b  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71  true false 13}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3b6aa044-fbdf-4221-a270-96bc83511b59:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3b6aa044-fbdf-4221-a270-96bc83511b59  true false 8}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-432c9e62-39b8-44ac-bf04-83e34b649318  true false 22}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-4c39d09b-b648-4b06-a903-e8005b8114ad:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4c39d09b-b648-4b06-a903-e8005b8114ad  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5b465967-3761-4776-94d8-06f5b348f6a0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5b465967-3761-4776-94d8-06f5b348f6a0  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-69b06f70-c974-4347-b57a-f9254747d08c  true false 4}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6  true false 3}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-947490b0-7614-49a1-a791-0edb87f913dd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-947490b0-7614-49a1-a791-0edb87f913dd  true false 28}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4  true false 5}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4  true false 23}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a  true false 16}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e  true false 26}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577  true false 24}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed215498-867b-4073-8a88-6dbf534db251  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba  true false 27})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:20:54.497197       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:20:54.497225       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:20:54.497318       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:54.497349       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:20:54.504223       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:20:54.504244       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-098c3baa-3153-4025-8102-780f961f253c","csi.storage.k8s.io/pvc/name":"pvc-ds984","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c"}
I0929 15:20:54.528571       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:20:54.528619       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.635693       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.635757       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.635786       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:20:54.635828       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.635916       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-24916d34-a3fa-440c-911d-d13576ee991b" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-24916d34-a3fa-440c-911d-d13576ee991b) in current disk list(len: 1) nor in diskMap(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c:0xc00094c780])
E0929 15:20:54.635941       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-24916d34-a3fa-440c-911d-d13576ee991b" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-24916d34-a3fa-440c-911d-d13576ee991b) in current disk list(len: 1) nor in diskMap(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c:0xc00094c780])
I0929 15:20:54.635966       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.636038       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.636069       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.636103       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.636120       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.636136       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.636175       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.636190       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.636214       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.636225       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.636239       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.636253       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.636268       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.636296       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.636310       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.636323       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.636370       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.636397       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.636409       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.636422       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.636445       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.636458       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.636470       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.636491       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.636502       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.636553       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.636579       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.636596       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.636611       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.636626       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.636639       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.636658       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-4c39d09b-b648-4b06-a903-e8005b8114ad" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-4c39d09b-b648-4b06-a903-e8005b8114ad) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.636678       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-4c39d09b-b648-4b06-a903-e8005b8114ad" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-4c39d09b-b648-4b06-a903-e8005b8114ad) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.636689       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.636730       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5b465967-3761-4776-94d8-06f5b348f6a0" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5b465967-3761-4776-94d8-06f5b348f6a0) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.636746       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5b465967-3761-4776-94d8-06f5b348f6a0" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5b465967-3761-4776-94d8-06f5b348f6a0) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.636792       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.636833       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.636854       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.636886       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.636903       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.636937       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.636961       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.636975       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.637000       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.637014       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.637034       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3b6aa044-fbdf-4221-a270-96bc83511b59" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3b6aa044-fbdf-4221-a270-96bc83511b59) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.637049       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3b6aa044-fbdf-4221-a270-96bc83511b59" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3b6aa044-fbdf-4221-a270-96bc83511b59) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.637070       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.637092       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.637109       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.637126       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.637139       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-947490b0-7614-49a1-a791-0edb87f913dd" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-947490b0-7614-49a1-a791-0edb87f913dd) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.637167       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-947490b0-7614-49a1-a791-0edb87f913dd" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-947490b0-7614-49a1-a791-0edb87f913dd) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.637179       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.637191       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.637205       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.637220       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.637246       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.637278       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.637299       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.637312       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.637326       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.637438       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.637481       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.637516       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.637562       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.637606       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.637632       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.637665       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.637691       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.637707       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.637748       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.637791       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.637713       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.637824       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.637870       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.637897       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.637936       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.637954       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.637901       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.638000       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.638032       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.638065       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.638093       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.638109       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.638133       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.638145       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:54.638159       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c) in current disk list(len: 1) nor in diskMap(map[])
E0929 15:20:54.638178       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c) in current disk list(len: 1) nor in diskMap(map[])
I0929 15:20:54.672948       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:20:54.672979       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92","csi.storage.k8s.io/pvc/name":"pvc-6gxj6","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92"}
I0929 15:20:54.672950       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:20:54.673094       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577","csi.storage.k8s.io/pvc/name":"pvc-zn99l","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577"}
I0929 15:20:54.673331       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:20:54.673351       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 40 lines ...
I0929 15:20:55.002448       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:55.002479       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:20:55.074999       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:20:55.075025       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:55.075045       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:20:55.173359       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:20:55.173396       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7f3f6a9a-3f43-4ea3-bddb-6b60fa2c5d92  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:20:55.173391       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e","csi.storage.k8s.io/pvc/name":"pvc-49xg5","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e"}
I0929 15:20:55.173762       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:55.173782       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:20:55.174066       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:20:55.174088       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:55.351420       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 35 lines ...
I0929 15:20:56.387250       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:20:56.550928       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:20:56.550954       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba","csi.storage.k8s.io/pvc/name":"pvc-xqfqc","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba"}
I0929 15:20:56.577175       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:20:56.577210       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:56.577232       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:20:56.694268       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-24916d34-a3fa-440c-911d-d13576ee991b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-24916d34-a3fa-440c-911d-d13576ee991b  true false 3}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a  true false 4}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24  true false 8}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b  true false 5})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:20:56.694472       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:56.694519       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:20:56.751799       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:20:56.751821       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-3b6aa044-fbdf-4221-a270-96bc83511b59","csi.storage.k8s.io/pvc/name":"pvc-5wbpj","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3b6aa044-fbdf-4221-a270-96bc83511b59"}
I0929 15:20:56.757292       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dffba81a-aea1-4b63-abfd-a4b8eee85c24" lun 8 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-098c3baa-3153-4025-8102-780f961f253c  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e  true false 16}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-4c39d09b-b648-4b06-a903-e8005b8114ad:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4c39d09b-b648-4b06-a903-e8005b8114ad  true false 13}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5b465967-3761-4776-94d8-06f5b348f6a0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5b465967-3761-4776-94d8-06f5b348f6a0  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba  true false 15})]
I0929 15:20:56.757348       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 18 lines ...
I0929 15:20:57.403702       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:20:57.551909       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:20:57.551935       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4","csi.storage.k8s.io/pvc/name":"pvc-tdkc2","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4"}
I0929 15:20:57.578881       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:20:57.578932       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:57.578981       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:20:57.641517       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-098c3baa-3153-4025-8102-780f961f253c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-098c3baa-3153-4025-8102-780f961f253c  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e  true false 16}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-4c39d09b-b648-4b06-a903-e8005b8114ad:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4c39d09b-b648-4b06-a903-e8005b8114ad  true false 13}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5b465967-3761-4776-94d8-06f5b348f6a0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5b465967-3761-4776-94d8-06f5b348f6a0  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a0d2b16a-764f-4a12-9c81-798d5ceb92e9  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e58aa0dd-e343-40be-a6cc-6166de3daf63  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc3ceae6-4fae-4556-945b-c78fb5e74eba  true false 15})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:20:57.641784       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:57.641835       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:20:57.736045       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030" lun 9 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3b6aa044-fbdf-4221-a270-96bc83511b59:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3b6aa044-fbdf-4221-a270-96bc83511b59  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-69b06f70-c974-4347-b57a-f9254747d08c  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-947490b0-7614-49a1-a791-0edb87f913dd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-947490b0-7614-49a1-a791-0edb87f913dd  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4  true false 23}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed215498-867b-4073-8a88-6dbf534db251  true false 22})]
I0929 15:20:57.736107       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:57.736182       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3b6aa044-fbdf-4221-a270-96bc83511b59:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3b6aa044-fbdf-4221-a270-96bc83511b59  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-69b06f70-c974-4347-b57a-f9254747d08c  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-947490b0-7614-49a1-a791-0edb87f913dd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-947490b0-7614-49a1-a791-0edb87f913dd  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4  true false 23}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed215498-867b-4073-8a88-6dbf534db251  true false 22})])
I0929 15:20:57.772252       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I0929 15:20:57.795665       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:20:57.957175       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:20:57.957261       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71","csi.storage.k8s.io/pvc/name":"pvc-p7j2r","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71"}
I0929 15:20:57.979131       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:20:57.979178       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:57.979223       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:20:58.012391       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3b6aa044-fbdf-4221-a270-96bc83511b59:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3b6aa044-fbdf-4221-a270-96bc83511b59  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-69b06f70-c974-4347-b57a-f9254747d08c  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-947490b0-7614-49a1-a791-0edb87f913dd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-947490b0-7614-49a1-a791-0edb87f913dd  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4  true false 23}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ed215498-867b-4073-8a88-6dbf534db251  true false 22})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:20:58.012597       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:58.012655       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:20:58.067744       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-9e6c84b5-6978-47cb-a9eb-988e26a933f4" lun 2 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e  true false 24})]
I0929 15:20:58.067785       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:58.067846       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e  true false 24})])
I0929 15:20:58.151476       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:20:58.151500       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-432c9e62-39b8-44ac-bf04-83e34b649318","csi.storage.k8s.io/pvc/name":"pvc-glnv8","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318"}
I0929 15:20:58.173457       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:20:58.173500       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:58.173548       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:20:58.291550       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-283501b9-1bc6-4c44-ab1a-c41854e70f71  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e  true false 24})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:20:58.291776       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:58.291854       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:20:58.346850       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-24916d34-a3fa-440c-911d-d13576ee991b" lun 3 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-432c9e62-39b8-44ac-bf04-83e34b649318  true false 26})]
I0929 15:20:58.346887       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:58.346946       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-432c9e62-39b8-44ac-bf04-83e34b649318  true false 26})])
I0929 15:20:58.372559       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:20:58.372582       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6","csi.storage.k8s.io/pvc/name":"pvc-nm8dz","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6"}
I0929 15:20:58.395751       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:20:58.395795       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:58.395858       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:20:58.551173       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:20:58.551202       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35","csi.storage.k8s.io/pvc/name":"pvc-g56gk","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35"}
I0929 15:20:58.573384       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-432c9e62-39b8-44ac-bf04-83e34b649318:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-432c9e62-39b8-44ac-bf04-83e34b649318  true false 26})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:20:58.573609       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:58.573663       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:20:58.574103       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:20:58.574165       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:58.694493       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a" lun 4 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6  true false 27})]
I0929 15:20:58.694567       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 20 lines ...
I0929 15:20:59.175905       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:20:59.175972       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:59.191295       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-76a88138-4681-46a5-998a-b01181972fdb" lun 0 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-76a88138-4681-46a5-998a-b01181972fdb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-76a88138-4681-46a5-998a-b01181972fdb  true false 0})]
I0929 15:20:59.191366       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:59.191429       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-76a88138-4681-46a5-998a-b01181972fdb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-76a88138-4681-46a5-998a-b01181972fdb  true false 0})])
I0929 15:20:59.191564       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:20:59.341377       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-76a88138-4681-46a5-998a-b01181972fdb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-76a88138-4681-46a5-998a-b01181972fdb  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:20:59.353424       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:20:59.353446       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738","csi.storage.k8s.io/pvc/name":"pvc-7c2vl","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738"}
I0929 15:20:59.374930       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:20:59.374974       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:59.375060       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:20:59.520584       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6  true false 27})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:20:59.520872       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:59.520906       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:20:59.551665       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:20:59.551687       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-5aa0369c-26f9-463b-98bd-ccd821476413","csi.storage.k8s.io/pvc/name":"test.csi.azure.comfrs88","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-5421","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413"}
I0929 15:20:59.582548       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413" to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:20:59.582590       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 2 lines ...
I0929 15:20:59.599007       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:20:59.599037       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-01886269-4737-4eda-9608-d73aa87e57ce  true false 29}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-12718de8-a30a-4b61-a208-606b8c0021d7  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35  true false 28})])
I0929 15:20:59.599070       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:59.599094       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413" lun 1 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5aa0369c-26f9-463b-98bd-ccd821476413  true false 1})]
I0929 15:20:59.599153       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:59.599170       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5aa0369c-26f9-463b-98bd-ccd821476413  true false 1})])
I0929 15:20:59.668019       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:20:59.668038       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:20:59.668087       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-01886269-4737-4eda-9608-d73aa87e57ce  true false 29}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-12718de8-a30a-4b61-a208-606b8c0021d7  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35  true false 28})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:20:59.668145       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-01886269-4737-4eda-9608-d73aa87e57ce  true false 29}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-12718de8-a30a-4b61-a208-606b8c0021d7  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35  true false 28})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:20:59.668210       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:20:59.668290       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:20:59.668387       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:59.668464       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:20:59.742667       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5aa0369c-26f9-463b-98bd-ccd821476413  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:20:59.758378       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc" lun 6 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[]
I0929 15:20:59.758410       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc" attached to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:20:59.758420       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-cf36cd8e-ecb8-43a7-8bb0-a551ef7d77dc" to node "k8s-agentpool1-18436497-vmss000001" successfully
I0929 15:20:59.758434       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"6"}}
I0929 15:20:59.758544       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:59.758587       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4" lun 11 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[]
... skipping 83 lines ...
I0929 15:20:59.760773       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:59.760809       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6" lun 27 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[]
I0929 15:20:59.760823       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6" attached to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:20:59.760832       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6" to node "k8s-agentpool1-18436497-vmss000001" successfully
I0929 15:20:59.760844       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"27"}}
I0929 15:20:59.760889       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:59.760918       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35) in current disk list(len: 28) nor in diskMap(map[])
E0929 15:20:59.760942       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35) in current disk list(len: 28) nor in diskMap(map[])
I0929 15:20:59.760956       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:59.761001       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce) in current disk list(len: 28) nor in diskMap(map[])
E0929 15:20:59.761021       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce) in current disk list(len: 28) nor in diskMap(map[])
I0929 15:20:59.761034       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:59.761063       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164) in current disk list(len: 28) nor in diskMap(map[])
E0929 15:20:59.761078       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164) in current disk list(len: 28) nor in diskMap(map[])
I0929 15:20:59.761089       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:59.761117       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7) in current disk list(len: 28) nor in diskMap(map[])
E0929 15:20:59.761132       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7) in current disk list(len: 28) nor in diskMap(map[])
I0929 15:20:59.761143       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E0929 15:20:59.761164       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738) in current disk list(len: 28) nor in diskMap(map[])
E0929 15:20:59.761185       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738) in current disk list(len: 28) nor in diskMap(map[])
I0929 15:20:59.952916       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:20:59.952944       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b","csi.storage.k8s.io/pvc/name":"pvc-pcfp4","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b"}
I0929 15:20:59.985718       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:20:59.985748       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:20:59.985815       1 azure_controller_common.go:457] azureDisk - find disk: lun 5 name "pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b"
I0929 15:20:59.985826       1 controllerserver.go:453] Attach operation is successful. volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b" is already attached to node "k8s-agentpool1-18436497-vmss000001" at lun 5.
... skipping 41 lines ...
I0929 15:21:02.375201       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:21:02.375287       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:21:02.375300       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:21:02.375329       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164" lun 28 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164  true false 28})]
I0929 15:21:02.375367       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:21:02.375399       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164  true false 28})])
I0929 15:21:02.566667       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3ca031ba-3182-4fba-a217-f3e83ed5c164  true false 28})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:21:02.753718       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:21:02.753745       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-01886269-4737-4eda-9608-d73aa87e57ce","csi.storage.k8s.io/pvc/name":"pvc-p2467","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce"}
I0929 15:21:02.785184       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:21:02.785211       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:21:02.785230       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:21:02.868025       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:21:02.868078       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:21:02.868095       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce" lun 29 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-01886269-4737-4eda-9608-d73aa87e57ce  true false 29})]
I0929 15:21:02.868111       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:21:02.868172       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-01886269-4737-4eda-9608-d73aa87e57ce  true false 29})])
I0929 15:21:03.031619       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-01886269-4737-4eda-9608-d73aa87e57ce:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-01886269-4737-4eda-9608-d73aa87e57ce  true false 29})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:21:03.150966       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:21:03.150996       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-69b06f70-c974-4347-b57a-f9254747d08c","csi.storage.k8s.io/pvc/name":"pvc-mgpgp","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c"}
I0929 15:21:03.190963       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:21:03.191007       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:21:03.191070       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:21:03.262986       1 azure_controller_common.go:457] azureDisk - find disk: lun 20 name "pvc-69b06f70-c974-4347-b57a-f9254747d08c" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c"
... skipping 5 lines ...
I0929 15:21:03.391052       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:21:03.391102       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:21:03.391132       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:21:03.391162       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7" lun 30 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-12718de8-a30a-4b61-a208-606b8c0021d7  true false 30})]
I0929 15:21:03.391207       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:21:03.391256       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-12718de8-a30a-4b61-a208-606b8c0021d7  true false 30})])
I0929 15:21:03.616464       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-12718de8-a30a-4b61-a208-606b8c0021d7  true false 30})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:21:04.088944       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I0929 15:21:04.088996       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dee7f437-d435-49ec-97ba-e4d0144153f4"}
I0929 15:21:04.089134       1 controllerserver.go:522] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dee7f437-d435-49ec-97ba-e4d0144153f4 from node k8s-agentpool1-18436497-vmss000001
I0929 15:21:04.089146       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:21:04.089168       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:21:04.155310       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 43 lines ...
I0929 15:21:18.980696       1 azure_controller_common.go:457] azureDisk - find disk: lun 27 name "pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6"
I0929 15:21:18.980708       1 controllerserver.go:453] Attach operation is successful. volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6" is already attached to node "k8s-agentpool1-18436497-vmss000001" at lun 27.
I0929 15:21:18.980726       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"27"}}
I0929 15:21:19.735745       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I0929 15:21:19.735771       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0"}
I0929 15:21:19.735846       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0)
I0929 15:21:19.735865       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0) returned with failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0) since it's in attaching or detaching state
E0929 15:21:19.735900       1 utils.go:100] GRPC error: failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0) since it's in attaching or detaching state
I0929 15:21:21.891635       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577" attached to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:21:21.891687       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577" to node "k8s-agentpool1-18436497-vmss000001" successfully
I0929 15:21:21.891722       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"7"}}
I0929 15:21:21.898323       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:21:21.898346       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577","csi.storage.k8s.io/pvc/name":"pvc-zn99l","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577"}
I0929 15:21:21.921231       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ec003d4b-4c63-4c85-9f0d-c5bf55738577" to node "k8s-agentpool1-18436497-vmss000001".
... skipping 94 lines ...
I0929 15:21:49.808743       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a","csi.storage.k8s.io/pvc/name":"pvc-899bg","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a"}
I0929 15:21:49.832704       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:21:49.832748       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:21:49.832768       1 azure_controller_common.go:457] azureDisk - find disk: lun 4 name "pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a"
I0929 15:21:49.832798       1 controllerserver.go:453] Attach operation is successful. volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a37b1ead-24bf-4dd1-a6b1-d29b691ab46a" is already attached to node "k8s-agentpool1-18436497-vmss000001" at lun 4.
I0929 15:21:49.832835       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"4"}}
I0929 15:21:50.010520       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35  true false 31})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:21:50.010709       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:21:50.010761       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:21:50.051332       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413" attached to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:21:50.051355       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413" to node "k8s-agentpool1-18436497-vmss000000" successfully
I0929 15:21:50.051387       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
I0929 15:21:50.068851       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-065e1ebf-4f2b-41c0-89ef-42e3bbabb738" lun 0 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[]
... skipping 5 lines ...
I0929 15:22:00.996535       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dee7f437-d435-49ec-97ba-e4d0144153f4)
I0929 15:22:06.221649       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dee7f437-d435-49ec-97ba-e4d0144153f4
I0929 15:22:06.221694       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dee7f437-d435-49ec-97ba-e4d0144153f4) returned with <nil>
I0929 15:22:06.221727       1 utils.go:102] GRPC response: {}
I0929 15:22:08.816113       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:22:08.816142       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8","parameters":{"csi.storage.k8s.io/pv/name":"pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8","csi.storage.k8s.io/pvc/name":"inline-volume-tester-9lmtp-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2509"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:22:08.836733       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:22:08.836832       1 controllerserver.go:269] begin to create azure disk(pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:22:08.836847       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:22:10.183857       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35" attached to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:22:10.183904       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35" to node "k8s-agentpool1-18436497-vmss000001" successfully
I0929 15:22:10.183925       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"31"}}
I0929 15:22:11.444421       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8 StorageAccountType:StandardSSD_LRS Size:5
... skipping 9 lines ...
I0929 15:22:12.262076       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:12.262144       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:22:12.262161       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:12.262172       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]
I0929 15:22:12.262207       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:12.262254       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})])
I0929 15:22:12.332564       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:12.332592       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:12.332676       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:12.332704       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:12.332761       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:12.332813       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:12.337742       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I0929 15:22:12.360822       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:22:12.425718       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:22:12.425778       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:12.425798       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]
I0929 15:22:12.425815       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:12.425842       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})])
I0929 15:22:12.493476       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:12.493507       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:12.493573       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:12.493618       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:12.493667       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:12.493698       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:12.499762       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I0929 15:22:12.521619       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:22:12.578819       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:22:12.578875       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:12.578892       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]
I0929 15:22:12.578909       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:12.578953       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})])
I0929 15:22:12.672267       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:12.672296       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:12.672417       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:12.672499       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:12.672535       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:12.672558       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:13.338168       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I0929 15:22:13.361065       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:22:13.439113       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:22:13.439173       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:13.439196       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]
I0929 15:22:13.439214       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:13.439243       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})])
I0929 15:22:13.503005       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:13.503036       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:13.503134       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:13.503203       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:13.503244       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:13.503281       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:18.852182       1 azure_vmss.go:319] Couldn't find VMSS VM with scaleSetName "k8s-agentpool1-18436497-vmss" and instanceID "1", refreshing the cache
... skipping 3 lines ...
I0929 15:22:21.542742       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:21.542841       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:22:21.542856       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:21.542868       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]
I0929 15:22:21.542911       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:21.542961       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})])
I0929 15:22:21.608160       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:21.608189       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:21.608269       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:21.608300       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:21.608406       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:21.608479       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:23.737374       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 3 lines ...
I0929 15:22:29.060862       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-449e2e3d-bd7a-4fd0-8118-ed8bd65db7f0) returned with <nil>
I0929 15:22:29.060898       1 utils.go:102] GRPC response: {}
I0929 15:22:33.158741       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:22:33.158768       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5","parameters":{"csi.storage.k8s.io/pv/name":"pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5","csi.storage.k8s.io/pvc/name":"inline-volume-tester-qlx6x-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-3038"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:22:33.159795       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:22:33.159818       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-6a356b69-0333-41f3-8bd1-21c780b2db34","parameters":{"csi.storage.k8s.io/pv/name":"pvc-6a356b69-0333-41f3-8bd1-21c780b2db34","csi.storage.k8s.io/pvc/name":"inline-volume-tester-qlx6x-my-volume-1","csi.storage.k8s.io/pvc/namespace":"ephemeral-3038"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:22:33.179043       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:22:33.179143       1 controllerserver.go:269] begin to create azure disk(pvc-6a356b69-0333-41f3-8bd1-21c780b2db34) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:22:33.179158       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-6a356b69-0333-41f3-8bd1-21c780b2db34 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:22:33.179384       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:22:33.179442       1 controllerserver.go:269] begin to create azure disk(pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:22:33.179461       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:22:35.543559       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:22:35.543618       1 controllerserver.go:346] create azure disk(pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5 kubernetes.io-created-for-pvc-name:inline-volume-tester-qlx6x-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-3038]) successfully
I0929 15:22:35.543662       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5","csi.storage.k8s.io/pvc/name":"inline-volume-tester-qlx6x-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-3038","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5"}}
I0929 15:22:35.549227       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-6a356b69-0333-41f3-8bd1-21c780b2db34 StorageAccountType:StandardSSD_LRS Size:5
... skipping 11 lines ...
I0929 15:22:36.366934       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:22:36.367008       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:36.367047       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})]
I0929 15:22:36.367065       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:36.367086       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:22:36.367120       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})])
I0929 15:22:36.428224       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:36.428244       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:36.428298       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:36.428321       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:36.428437       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:36.428483       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:36.428564       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 3 lines ...
I0929 15:22:36.458464       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:22:36.458487       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:36.515116       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})]
I0929 15:22:36.515145       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:36.515212       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})])
I0929 15:22:36.515212       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:22:36.599767       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:36.599789       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:36.599845       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:36.599894       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:36.599950       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:36.599989       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:36.600069       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 3 lines ...
I0929 15:22:36.629366       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:22:36.629392       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:36.651172       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})]
I0929 15:22:36.651196       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:36.651243       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})])
I0929 15:22:36.651480       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:22:36.713306       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:36.713326       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:36.713374       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:36.713410       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:36.713456       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:36.713516       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:36.713575       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:36.713594       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:22:36.780941       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})]
I0929 15:22:36.780989       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:36.781038       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})])
I0929 15:22:36.884934       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:36.884967       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:36.885034       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:36.885089       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:36.885132       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:36.885163       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:37.436116       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 7 lines ...
I0929 15:22:37.561149       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:37.561174       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})])
I0929 15:22:37.608207       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:22:37.608243       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5","csi.storage.k8s.io/pvc/name":"inline-volume-tester-qlx6x-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-3038","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5"}
I0929 15:22:37.617997       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:22:37.618018       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8","csi.storage.k8s.io/pvc/name":"inline-volume-tester-9lmtp-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2509","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8"}
I0929 15:22:37.622631       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:37.622675       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:37.622737       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:37.622821       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:37.622887       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:37.622935       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:37.635460       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to node "k8s-agentpool1-18436497-vmss000001".
... skipping 4 lines ...
I0929 15:22:37.740655       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:22:37.740726       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:37.740742       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})]
I0929 15:22:37.740755       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:37.740782       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})])
I0929 15:22:37.740744       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:22:37.802248       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:37.802274       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:37.802368       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:37.802419       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:37.802462       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:37.802527       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:37.802584       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:37.802606       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:22:37.853870       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]
I0929 15:22:37.853903       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:37.853960       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})])
I0929 15:22:37.941359       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:37.941386       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:37.941450       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:37.941502       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:37.941534       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:37.941582       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:41.632984       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I0929 15:22:41.655384       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:22:41.713494       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:22:41.713554       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:41.713575       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})]
I0929 15:22:41.713592       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:41.713639       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})])
I0929 15:22:41.793006       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:41.793034       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:41.793080       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:41.793114       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:41.793177       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:41.793253       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:41.811312       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I0929 15:22:41.841009       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:22:41.897064       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:22:41.897115       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:41.897135       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})]
I0929 15:22:41.897166       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:41.897212       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})])
I0929 15:22:41.957958       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:41.957988       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:41.958035       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:41.958069       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:41.958243       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:41.958278       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:49.805650       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 8 lines ...
I0929 15:22:49.913052       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})])
I0929 15:22:49.966676       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:22:49.966697       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5","csi.storage.k8s.io/pvc/name":"inline-volume-tester-qlx6x-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-3038","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5"}
I0929 15:22:49.991597       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:22:49.991660       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:49.991732       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:22:49.997941       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:49.997963       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:49.998034       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:49.998058       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:49.998134       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:49.998182       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:49.998235       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:49.998257       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:22:50.049256       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})]
I0929 15:22:50.049292       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:22:50.049323       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})])
I0929 15:22:50.134539       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:22:50.134606       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:22:50.134683       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:22:50.134720       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:22:50.134780       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:22:50.134826       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:23:04.697496       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
... skipping 13 lines ...
I0929 15:23:06.096327       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:06.096348       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})]
I0929 15:23:06.096411       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:06.096446       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})])
I0929 15:23:06.149844       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:23:06.149885       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5","csi.storage.k8s.io/pvc/name":"inline-volume-tester-qlx6x-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-3038","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5"}
I0929 15:23:06.174593       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:23:06.174621       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:23:06.174686       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:23:06.174746       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:23:06.174802       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:23:06.174839       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:23:06.198240       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:23:06.198275       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:06.198296       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:23:06.249558       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:23:06.249815       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:06.249848       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})]
I0929 15:23:06.249869       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:06.249906       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})])
I0929 15:23:06.325716       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:23:06.325745       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:23:06.325791       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:23:06.325841       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:23:06.325895       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:23:06.325946       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:23:09.948779       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I0929 15:23:09.973375       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:23:10.062593       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:23:10.062638       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:10.062659       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]
I0929 15:23:10.062676       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:10.062713       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})])
I0929 15:23:10.126618       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:23:10.126646       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:23:10.126735       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:23:10.126785       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:23:10.126837       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:23:10.126869       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:23:13.431803       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 28 lines ...
I0929 15:23:17.317645       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:23:17.317687       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:17.317751       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:23:17.331573       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:23:17.331597       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:17.331666       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:23:17.367801       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:23:17.367830       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:23:17.367903       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:23:17.367930       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:23:17.367993       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:23:17.368044       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:23:17.368139       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:17.368173       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:23:17.423552       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 33})]
I0929 15:23:17.423599       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:17.423654       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 33})])
I0929 15:23:17.513211       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:23:17.513245       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:23:17.513296       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 33})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:23:17.513337       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 33})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:23:17.513392       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:23:17.513472       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:23:17.513501       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:17.513537       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
E0929 15:23:17.567447       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8) in current disk list(len: 32) nor in diskMap(map[])
E0929 15:23:17.567507       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8) in current disk list(len: 32) nor in diskMap(map[])
I0929 15:23:18.638575       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb
I0929 15:23:18.638608       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed8cc6d1-80f6-48fd-94e9-71cbae5426bb) returned with <nil>
I0929 15:23:18.638660       1 utils.go:102] GRPC response: {}
I0929 15:23:21.144244       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:23:21.144270       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3","parameters":{"csi.storage.k8s.io/pv/name":"pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3","csi.storage.k8s.io/pvc/name":"test.csi.azure.com6sstc","csi.storage.k8s.io/pvc/namespace":"provisioning-3027"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:23:21.191117       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:23:21.191212       1 controllerserver.go:269] begin to create azure disk(pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:23:21.191226       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:23:23.494042       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:23:23.494099       1 controllerserver.go:346] create azure disk(pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3 kubernetes.io-created-for-pvc-name:test.csi.azure.com6sstc kubernetes.io-created-for-pvc-namespace:provisioning-3027]) successfully
I0929 15:23:23.494123       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3","csi.storage.k8s.io/pvc/name":"test.csi.azure.com6sstc","csi.storage.k8s.io/pvc/namespace":"provisioning-3027","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3"}}
I0929 15:23:25.240101       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-76a88138-4681-46a5-998a-b01181972fdb:pvc-76a88138-4681-46a5-998a-b01181972fdb]) returned with <nil>
... skipping 7 lines ...
I0929 15:23:25.860700       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000000, refreshing the cache
I0929 15:23:25.930087       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:23:25.930134       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:25.930156       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3" lun 0 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3  true false 0})]
I0929 15:23:25.930174       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:25.930187       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3  true false 0})])
I0929 15:23:26.192562       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:23:27.703636       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I0929 15:23:27.703664       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-76a88138-4681-46a5-998a-b01181972fdb"}
I0929 15:23:27.703744       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-76a88138-4681-46a5-998a-b01181972fdb)
I0929 15:23:28.381421       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I0929 15:23:28.381451       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413"}
I0929 15:23:28.381533       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413)
... skipping 2 lines ...
I0929 15:23:32.917967       1 utils.go:102] GRPC response: {}
I0929 15:23:33.589767       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413
I0929 15:23:33.589812       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5aa0369c-26f9-463b-98bd-ccd821476413) returned with <nil>
I0929 15:23:33.589853       1 utils.go:102] GRPC response: {}
I0929 15:23:36.181174       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:23:36.181221       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d","parameters":{"csi.storage.k8s.io/pv/name":"pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d","csi.storage.k8s.io/pvc/name":"pvc-h6ln5","csi.storage.k8s.io/pvc/namespace":"provisioning-7684"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:23:36.200356       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:23:36.200489       1 controllerserver.go:269] begin to create azure disk(pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:23:36.200504       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d StorageAccountType:StandardSSD_LRS Size:5
I0929 15:23:36.302912       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3" attached to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:23:36.302945       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3" to node "k8s-agentpool1-18436497-vmss000000" successfully
I0929 15:23:36.302982       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I0929 15:23:36.308559       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 12 lines ...
I0929 15:23:38.208619       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:38.208649       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})]
I0929 15:23:38.208672       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:38.208704       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})])
I0929 15:23:38.334705       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:23:38.334746       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5","csi.storage.k8s.io/pvc/name":"inline-volume-tester-qlx6x-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-3038","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5"}
I0929 15:23:38.356509       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:23:38.356531       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:23:38.356605       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:23:38.356650       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:23:38.356704       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:23:38.356751       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:23:38.378231       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:23:38.378258       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:38.378291       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:23:38.442182       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:23:38.442256       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:38.442275       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})]
I0929 15:23:38.442291       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:38.442337       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})])
I0929 15:23:38.519645       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:23:38.519671       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:23:38.519721       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:23:38.519788       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:23:38.519817       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:23:38.519857       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:23:38.592071       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d StorageAccountType:StandardSSD_LRS Size:5
I0929 15:23:38.592157       1 controllerserver.go:346] create azure disk(pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d kubernetes.io-created-for-pvc-name:pvc-h6ln5 kubernetes.io-created-for-pvc-namespace:provisioning-7684]) successfully
I0929 15:23:38.592199       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d","csi.storage.k8s.io/pvc/name":"pvc-h6ln5","csi.storage.k8s.io/pvc/namespace":"provisioning-7684","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d"}}
I0929 15:23:39.879604       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:23:39.879629       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d","parameters":{"csi.storage.k8s.io/pv/name":"pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d","csi.storage.k8s.io/pvc/name":"test.csi.azure.comv42d9","csi.storage.k8s.io/pvc/namespace":"volumemode-1860"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:23:39.900618       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:23:39.900712       1 controllerserver.go:269] begin to create azure disk(pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:23:39.900760       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d StorageAccountType:StandardSSD_LRS Size:5
I0929 15:23:41.281127       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:23:41.281153       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d","csi.storage.k8s.io/pvc/name":"pvc-h6ln5","csi.storage.k8s.io/pvc/namespace":"provisioning-7684","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d"}
I0929 15:23:41.304106       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d" to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:23:41.304148       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:41.304184       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:23:41.304532       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:41.304568       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d" lun 1 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d  true false 1})]
I0929 15:23:41.304597       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:41.304617       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d  true false 1})])
I0929 15:23:41.456194       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:23:42.213331       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d StorageAccountType:StandardSSD_LRS Size:5
I0929 15:23:42.213391       1 controllerserver.go:346] create azure disk(pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d kubernetes.io-created-for-pvc-name:test.csi.azure.comv42d9 kubernetes.io-created-for-pvc-namespace:volumemode-1860]) successfully
I0929 15:23:42.213421       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d","csi.storage.k8s.io/pvc/name":"test.csi.azure.comv42d9","csi.storage.k8s.io/pvc/namespace":"volumemode-1860","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d"}}
I0929 15:23:44.610333       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:23:44.610362       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d","csi.storage.k8s.io/pvc/name":"test.csi.azure.comv42d9","csi.storage.k8s.io/pvc/namespace":"volumemode-1860","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d"}
I0929 15:23:44.688261       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d" to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:23:44.688296       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:44.688319       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000000, refreshing the cache
I0929 15:23:44.822801       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:23:44.822843       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:44.822864       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d" lun 2 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d  true false 2})]
I0929 15:23:44.822881       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:44.822894       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d  true false 2})])
I0929 15:23:45.041065       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:23:55.015696       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I0929 15:23:55.015723       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000000","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3"}
I0929 15:23:55.015840       1 controllerserver.go:522] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3 from node k8s-agentpool1-18436497-vmss000000
I0929 15:23:55.015859       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:23:55.015883       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000000, refreshing the cache
I0929 15:23:55.087505       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 38 lines ...
I0929 15:24:10.483758       1 azure_controller_vmss.go:174] azureDisk - detach disk: name "pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d"
I0929 15:24:10.483781       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d:pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d])
I0929 15:24:10.483949       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:24:10.512198       1 utils.go:102] GRPC response: {"snapshot":{"creation_time":{"nanos":320337500,"seconds":1632929048},"ready_to_use":true,"size_bytes":5368709120,"snapshot_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/snapshots/snapshot-3eeb53a6-3580-4920-aac9-2a2dd539f552","source_volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d"}}
I0929 15:24:12.893928       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:24:12.893956       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-cc8faaca-1016-49b1-ba5d-4429d129798e","parameters":{"csi.storage.k8s.io/pv/name":"pvc-cc8faaca-1016-49b1-ba5d-4429d129798e","csi.storage.k8s.io/pvc/name":"pvc-t8twz","csi.storage.k8s.io/pvc/namespace":"provisioning-7684"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}],"volume_content_source":{"Type":{"Snapshot":{"snapshot_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/snapshots/snapshot-3eeb53a6-3580-4920-aac9-2a2dd539f552"}}}}
I0929 15:24:12.919935       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-cc8faaca-1016-49b1-ba5d-4429d129798e, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-cc8faaca-1016-49b1-ba5d-4429d129798e' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:24:12.920004       1 controllerserver.go:269] begin to create azure disk(pvc-cc8faaca-1016-49b1-ba5d-4429d129798e) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:24:12.920035       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-cc8faaca-1016-49b1-ba5d-4429d129798e StorageAccountType:StandardSSD_LRS Size:5
I0929 15:24:14.143173       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:24:14.143200       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8","csi.storage.k8s.io/pvc/name":"inline-volume-tester-9lmtp-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2509","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8"}
I0929 15:24:14.167062       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:24:14.167095       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:24:14.167136       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:24:14.167154       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:24:14.167175       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" lun 32 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]
I0929 15:24:14.167200       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:24:14.167233       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})])
I0929 15:24:14.248546       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:24:14.248591       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-18436497-vmss/virtualMachines/1, error: %!s(<nil>)
E0929 15:24:14.248657       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})]) on rg(kubetest-xhnqwtqz) vm(k8s-agentpool1-18436497-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I0929 15:24:14.248685       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E0929 15:24:14.248768       1 controllerserver.go:479] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E0929 15:24:14.248801       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to instance "k8s-agentpool1-18436497-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I0929 15:24:15.160681       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
... skipping 18 lines ...
I0929 15:24:20.697511       1 utils.go:102] GRPC response: {}
I0929 15:24:20.762798       1 azure_controller_vmss.go:174] azureDisk - detach disk: name "pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d"
I0929 15:24:20.762822       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d:pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3:pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3])
I0929 15:24:20.993053       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I0929 15:24:20.993081       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3"}
I0929 15:24:20.993173       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3)
I0929 15:24:20.993206       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3) returned with failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3) since it's in attaching or detaching state
E0929 15:24:20.993224       1 utils.go:100] GRPC error: failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-a54aa0c7-6b4d-4cbd-aa27-1fb11c2119f3) since it's in attaching or detaching state
I0929 15:24:25.580134       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I0929 15:24:25.580162       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d"}
I0929 15:24:25.580247       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d)
I0929 15:24:30.785751       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d
I0929 15:24:30.785794       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-bd964e49-c36c-4ed0-afe9-03bd1427212d) returned with <nil>
I0929 15:24:30.785811       1 utils.go:102] GRPC response: {}
... skipping 8 lines ...
I0929 15:24:36.159122       1 controllerserver.go:531] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-1b4a5575-a669-4ff7-bcae-15b9fa64f36d from node k8s-agentpool1-18436497-vmss000000 successfully
I0929 15:24:36.159160       1 utils.go:102] GRPC response: {}
I0929 15:24:36.159250       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:24:36.159271       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-cc8faaca-1016-49b1-ba5d-4429d129798e" lun 0 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-cc8faaca-1016-49b1-ba5d-4429d129798e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc8faaca-1016-49b1-ba5d-4429d129798e  true false 0})]
I0929 15:24:36.159298       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:24:36.159330       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-cc8faaca-1016-49b1-ba5d-4429d129798e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc8faaca-1016-49b1-ba5d-4429d129798e  true false 0})])
I0929 15:24:36.318953       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-cc8faaca-1016-49b1-ba5d-4429d129798e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc8faaca-1016-49b1-ba5d-4429d129798e  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:24:39.230606       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:24:39.230635       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-94599083-7868-432e-a281-debe5ff820e0","parameters":{"csi.storage.k8s.io/pv/name":"pvc-94599083-7868-432e-a281-debe5ff820e0","csi.storage.k8s.io/pvc/name":"inline-volume-tester-sjhg5-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-9197"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:24:39.249135       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-94599083-7868-432e-a281-debe5ff820e0, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-94599083-7868-432e-a281-debe5ff820e0' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:24:39.249308       1 controllerserver.go:269] begin to create azure disk(pvc-94599083-7868-432e-a281-debe5ff820e0) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:24:39.249335       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-94599083-7868-432e-a281-debe5ff820e0 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:24:41.561966       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-94599083-7868-432e-a281-debe5ff820e0 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:24:41.562039       1 controllerserver.go:346] create azure disk(pvc-94599083-7868-432e-a281-debe5ff820e0) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-94599083-7868-432e-a281-debe5ff820e0 kubernetes.io-created-for-pvc-name:inline-volume-tester-sjhg5-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-9197]) successfully
I0929 15:24:41.562078       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-94599083-7868-432e-a281-debe5ff820e0","csi.storage.k8s.io/pvc/name":"inline-volume-tester-sjhg5-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-9197","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-94599083-7868-432e-a281-debe5ff820e0"}}
I0929 15:24:42.313410       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I0929 15:24:42.337990       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000000, refreshing the cache
I0929 15:24:42.407390       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-94599083-7868-432e-a281-debe5ff820e0" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:24:42.407428       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:24:42.407447       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-94599083-7868-432e-a281-debe5ff820e0" lun 1 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-94599083-7868-432e-a281-debe5ff820e0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-94599083-7868-432e-a281-debe5ff820e0  true false 1})]
I0929 15:24:42.407464       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:24:42.407477       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-94599083-7868-432e-a281-debe5ff820e0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-94599083-7868-432e-a281-debe5ff820e0  true false 1})])
I0929 15:24:42.576714       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-94599083-7868-432e-a281-debe5ff820e0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-94599083-7868-432e-a281-debe5ff820e0  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:24:51.424376       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-cc8faaca-1016-49b1-ba5d-4429d129798e" attached to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:24:51.424413       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-cc8faaca-1016-49b1-ba5d-4429d129798e" to node "k8s-agentpool1-18436497-vmss000000" successfully
I0929 15:24:51.424429       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I0929 15:24:52.685078       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-94599083-7868-432e-a281-debe5ff820e0" attached to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:24:52.685110       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-94599083-7868-432e-a281-debe5ff820e0" to node "k8s-agentpool1-18436497-vmss000000" successfully
I0929 15:24:52.685145       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
... skipping 408 lines ...
I0929 15:26:11.612292       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:26:11.612311       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 0})])
I0929 15:26:11.688882       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-cc8faaca-1016-49b1-ba5d-4429d129798e:pvc-cc8faaca-1016-49b1-ba5d-4429d129798e]) returned with <nil>
I0929 15:26:11.688937       1 azure_controller_common.go:369] azureDisk - detach disk(pvc-cc8faaca-1016-49b1-ba5d-4429d129798e, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-cc8faaca-1016-49b1-ba5d-4429d129798e) succeeded
I0929 15:26:11.688947       1 controllerserver.go:531] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-cc8faaca-1016-49b1-ba5d-4429d129798e from node k8s-agentpool1-18436497-vmss000000 successfully
I0929 15:26:11.688955       1 utils.go:102] GRPC response: {}
I0929 15:26:11.800840       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6a356b69-0333-41f3-8bd1-21c780b2db34  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:26:11.801026       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:26:11.801051       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:26:11.848509       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" lun 1 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[]
I0929 15:26:11.848560       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" attached to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:26:11.848571       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5" to node "k8s-agentpool1-18436497-vmss000001" successfully
I0929 15:26:11.848586       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
... skipping 133 lines ...
I0929 15:26:26.777183       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I0929 15:26:26.777210       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4"}
I0929 15:26:26.777309       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4)
I0929 15:26:26.908463       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I0929 15:26:26.908490       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c"}
I0929 15:26:26.908573       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c)
I0929 15:26:26.908586       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c) returned with failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c) since it's in attaching or detaching state
E0929 15:26:26.908612       1 utils.go:100] GRPC error: failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c) since it's in attaching or detaching state
I0929 15:26:27.117617       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I0929 15:26:27.117647       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7"}
I0929 15:26:27.117736       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-12718de8-a30a-4b61-a208-606b8c0021d7)
I0929 15:26:27.221277       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-24916d34-a3fa-440c-911d-d13576ee991b:pvc-24916d34-a3fa-440c-911d-d13576ee991b /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4:pvc-68ef8e0b-4169-47db-b1f1-486bd9654ed4 /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c:pvc-69b06f70-c974-4347-b57a-f9254747d08c /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e:pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35:pvc-df9602ec-31c8-471d-b9f4-fbfc39714c35]) returned with <nil>
I0929 15:26:27.221346       1 azure_controller_common.go:369] azureDisk - detach disk(pvc-69b06f70-c974-4347-b57a-f9254747d08c, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c) succeeded
I0929 15:26:27.221367       1 controllerserver.go:531] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-69b06f70-c974-4347-b57a-f9254747d08c from node k8s-agentpool1-18436497-vmss000001 successfully
... skipping 169 lines ...
I0929 15:28:30.282369       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:28:30.282411       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:28:30.282427       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:28:30.282439       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" lun 2 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 2})]
I0929 15:28:30.282455       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:28:30.282489       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 2})])
I0929 15:28:30.450275       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:28:45.563837       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" attached to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:28:45.563991       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8" to node "k8s-agentpool1-18436497-vmss000001" successfully
I0929 15:28:45.564042       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"2"}}
I0929 15:29:19.703690       1 azure_vmss.go:319] Couldn't find VMSS VM with scaleSetName "k8s-agentpool1-18436497-vmss" and instanceID "1", refreshing the cache
I0929 15:29:27.138878       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I0929 15:29:27.138909       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5cbf8152-a37f-403e-85e1-4fc61a57bfe5"}
... skipping 32 lines ...
I0929 15:30:29.896507       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34)
I0929 15:30:35.144745       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34
I0929 15:30:35.144779       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6a356b69-0333-41f3-8bd1-21c780b2db34) returned with <nil>
I0929 15:30:35.144796       1 utils.go:102] GRPC response: {}
I0929 15:32:35.269292       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:32:35.269319       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174","parameters":{"csi.storage.k8s.io/pv/name":"pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-hjdm2-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2509"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:32:35.289164       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:32:35.289245       1 controllerserver.go:269] begin to create azure disk(pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) diskZone() maxShares(0)
I0929 15:32:35.289264       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:32:37.605776       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174 StorageAccountType:StandardSSD_LRS Size:5
I0929 15:32:37.605829       1 controllerserver.go:346] create azure disk(pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174 kubernetes.io-created-for-pvc-name:inline-volume-tester2-hjdm2-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-2509]) successfully
I0929 15:32:37.605852       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-hjdm2-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2509","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174"}}
I0929 15:32:38.597585       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 2 lines ...
I0929 15:32:38.619742       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:32:38.619798       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:32:38.619812       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:32:38.619823       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174" lun 0 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174  true false 0})]
I0929 15:32:38.619853       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:32:38.619883       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174  true false 0})])
I0929 15:32:39.409566       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:32:49.535830       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174" attached to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:32:49.535883       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174" to node "k8s-agentpool1-18436497-vmss000000" successfully
I0929 15:32:49.535901       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I0929 15:33:20.181164       1 azure_vmss.go:319] Couldn't find VMSS VM with scaleSetName "k8s-agentpool1-18436497-vmss" and instanceID "0", refreshing the cache
I0929 15:33:38.859038       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I0929 15:33:38.859065       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000000","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-6030b6d5-9f81-49fc-b0c4-e21deaf0f174"}
... skipping 43 lines ...
I0929 15:35:12.750966       1 controllerserver.go:387] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8)
I0929 15:35:17.964260       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8
I0929 15:35:17.964289       1 controllerserver.go:389] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-80ee3f70-48bc-42ee-bd11-ad67a065c8d8) returned with <nil>
I0929 15:35:17.964314       1 utils.go:102] GRPC response: {}
I0929 15:35:22.012377       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:35:22.012423       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-5e85419d-4e97-4c38-885b-4da545c28ff4","parameters":{"csi.storage.k8s.io/pv/name":"pvc-5e85419d-4e97-4c38-885b-4da545c28ff4","csi.storage.k8s.io/pvc/name":"pvc-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","skuname":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:35:22.041853       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5e85419d-4e97-4c38-885b-4da545c28ff4, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-5e85419d-4e97-4c38-885b-4da545c28ff4' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:35:22.041978       1 controllerserver.go:269] begin to create azure disk(pvc-5e85419d-4e97-4c38-885b-4da545c28ff4) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(10) diskZone() maxShares(0)
I0929 15:35:22.042012       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-5e85419d-4e97-4c38-885b-4da545c28ff4 StorageAccountType:StandardSSD_LRS Size:10
I0929 15:35:24.368916       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-5e85419d-4e97-4c38-885b-4da545c28ff4 StorageAccountType:StandardSSD_LRS Size:10
I0929 15:35:24.368976       1 controllerserver.go:346] create azure disk(pvc-5e85419d-4e97-4c38-885b-4da545c28ff4) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-5e85419d-4e97-4c38-885b-4da545c28ff4 kubernetes.io-created-for-pvc-name:pvc-azuredisk kubernetes.io-created-for-pvc-namespace:default]) successfully
I0929 15:35:24.369000       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-5e85419d-4e97-4c38-885b-4da545c28ff4","csi.storage.k8s.io/pvc/name":"pvc-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuname":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5e85419d-4e97-4c38-885b-4da545c28ff4"}}
I0929 15:35:26.777032       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 2 lines ...
I0929 15:35:26.822000       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:35:26.822041       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5e85419d-4e97-4c38-885b-4da545c28ff4" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:35:26.822078       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:35:26.822106       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5e85419d-4e97-4c38-885b-4da545c28ff4" lun 0 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5e85419d-4e97-4c38-885b-4da545c28ff4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5e85419d-4e97-4c38-885b-4da545c28ff4  true false 0})]
I0929 15:35:26.822122       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:35:26.822153       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5e85419d-4e97-4c38-885b-4da545c28ff4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5e85419d-4e97-4c38-885b-4da545c28ff4  true false 0})])
I0929 15:35:26.987620       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-5e85419d-4e97-4c38-885b-4da545c28ff4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5e85419d-4e97-4c38-885b-4da545c28ff4  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:35:37.089884       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5e85419d-4e97-4c38-885b-4da545c28ff4" attached to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:35:37.089927       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-5e85419d-4e97-4c38-885b-4da545c28ff4" to node "k8s-agentpool1-18436497-vmss000000" successfully
I0929 15:35:37.089946       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I0929 15:35:50.742660       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:35:50.742700       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-da267650-3420-45bb-8d76-0168cfb4ddbb","parameters":{"csi.storage.k8s.io/pv/name":"pvc-da267650-3420-45bb-8d76-0168cfb4ddbb","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-0","csi.storage.k8s.io/pvc/namespace":"default","skuname":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:35:50.764770       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-da267650-3420-45bb-8d76-0168cfb4ddbb, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-da267650-3420-45bb-8d76-0168cfb4ddbb' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:35:50.764878       1 controllerserver.go:269] begin to create azure disk(pvc-da267650-3420-45bb-8d76-0168cfb4ddbb) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(10) diskZone() maxShares(0)
I0929 15:35:50.764966       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-da267650-3420-45bb-8d76-0168cfb4ddbb StorageAccountType:StandardSSD_LRS Size:10
I0929 15:35:53.080461       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-da267650-3420-45bb-8d76-0168cfb4ddbb StorageAccountType:StandardSSD_LRS Size:10
I0929 15:35:53.080503       1 controllerserver.go:346] create azure disk(pvc-da267650-3420-45bb-8d76-0168cfb4ddbb) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-da267650-3420-45bb-8d76-0168cfb4ddbb kubernetes.io-created-for-pvc-name:persistent-storage-statefulset-azuredisk-0 kubernetes.io-created-for-pvc-namespace:default]) successfully
I0929 15:35:53.080546       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-da267650-3420-45bb-8d76-0168cfb4ddbb","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuname":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-da267650-3420-45bb-8d76-0168cfb4ddbb"}}
I0929 15:35:55.756164       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I0929 15:35:56.401036       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:35:56.468131       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-da267650-3420-45bb-8d76-0168cfb4ddbb" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:35:56.468184       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:35:56.468217       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-da267650-3420-45bb-8d76-0168cfb4ddbb" lun 0 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-da267650-3420-45bb-8d76-0168cfb4ddbb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-da267650-3420-45bb-8d76-0168cfb4ddbb  true false 0})]
I0929 15:35:56.468232       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:35:56.468243       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-da267650-3420-45bb-8d76-0168cfb4ddbb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-da267650-3420-45bb-8d76-0168cfb4ddbb  true false 0})])
I0929 15:35:56.641239       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-da267650-3420-45bb-8d76-0168cfb4ddbb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-da267650-3420-45bb-8d76-0168cfb4ddbb  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:36:06.773184       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-da267650-3420-45bb-8d76-0168cfb4ddbb" attached to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:36:06.773237       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-da267650-3420-45bb-8d76-0168cfb4ddbb" to node "k8s-agentpool1-18436497-vmss000001" successfully
I0929 15:36:06.773273       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I0929 15:36:20.398143       1 azure_vmss.go:319] Couldn't find VMSS VM with scaleSetName "k8s-agentpool1-18436497-vmss" and instanceID "1", refreshing the cache
I0929 15:36:22.451710       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:36:22.451738       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-d43535af-0e91-4d9f-8310-0862a605a16b","parameters":{"csi.storage.k8s.io/pv/name":"pvc-d43535af-0e91-4d9f-8310-0862a605a16b","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-nonroot-0","csi.storage.k8s.io/pvc/namespace":"default","skuname":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:36:22.479687       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d43535af-0e91-4d9f-8310-0862a605a16b, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-d43535af-0e91-4d9f-8310-0862a605a16b' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:36:22.479786       1 controllerserver.go:269] begin to create azure disk(pvc-d43535af-0e91-4d9f-8310-0862a605a16b) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(10) diskZone() maxShares(0)
I0929 15:36:22.479802       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-d43535af-0e91-4d9f-8310-0862a605a16b StorageAccountType:StandardSSD_LRS Size:10
I0929 15:36:24.790125       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-d43535af-0e91-4d9f-8310-0862a605a16b StorageAccountType:StandardSSD_LRS Size:10
I0929 15:36:24.790179       1 controllerserver.go:346] create azure disk(pvc-d43535af-0e91-4d9f-8310-0862a605a16b) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-d43535af-0e91-4d9f-8310-0862a605a16b kubernetes.io-created-for-pvc-name:persistent-storage-statefulset-azuredisk-nonroot-0 kubernetes.io-created-for-pvc-namespace:default]) successfully
I0929 15:36:24.790203       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d43535af-0e91-4d9f-8310-0862a605a16b","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-nonroot-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuname":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d43535af-0e91-4d9f-8310-0862a605a16b"}}
I0929 15:36:27.850285       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 2 lines ...
I0929 15:36:27.875109       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:36:27.875148       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d43535af-0e91-4d9f-8310-0862a605a16b" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:36:27.875165       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:36:27.875192       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d43535af-0e91-4d9f-8310-0862a605a16b" lun 1 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-d43535af-0e91-4d9f-8310-0862a605a16b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d43535af-0e91-4d9f-8310-0862a605a16b  true false 1})]
I0929 15:36:27.875225       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:36:27.875254       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-d43535af-0e91-4d9f-8310-0862a605a16b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d43535af-0e91-4d9f-8310-0862a605a16b  true false 1})])
I0929 15:36:28.054170       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-d43535af-0e91-4d9f-8310-0862a605a16b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d43535af-0e91-4d9f-8310-0862a605a16b  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:36:38.163520       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d43535af-0e91-4d9f-8310-0862a605a16b" attached to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:36:38.163554       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d43535af-0e91-4d9f-8310-0862a605a16b" to node "k8s-agentpool1-18436497-vmss000000" successfully
I0929 15:36:38.163610       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
I0929 15:36:38.178910       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:36:38.178941       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d43535af-0e91-4d9f-8310-0862a605a16b","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-nonroot-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuname":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d43535af-0e91-4d9f-8310-0862a605a16b"}
I0929 15:36:38.203246       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d43535af-0e91-4d9f-8310-0862a605a16b" to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:36:38.203273       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:36:38.203315       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000000, refreshing the cache
I0929 15:36:38.269618       1 azure_controller_common.go:457] azureDisk - find disk: lun 1 name "pvc-d43535af-0e91-4d9f-8310-0862a605a16b" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d43535af-0e91-4d9f-8310-0862a605a16b"
I0929 15:36:38.269665       1 controllerserver.go:453] Attach operation is successful. volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-d43535af-0e91-4d9f-8310-0862a605a16b" is already attached to node "k8s-agentpool1-18436497-vmss000000" at lun 1.
I0929 15:36:38.269681       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
I0929 15:36:51.633416       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:36:51.633443       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029","parameters":{"csi.storage.k8s.io/pv/name":"pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029","csi.storage.k8s.io/pvc/name":"nginx-azuredisk-ephemeral-azuredisk01","csi.storage.k8s.io/pvc/namespace":"default","skuname":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:36:51.655352       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:36:51.655482       1 controllerserver.go:269] begin to create azure disk(pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(10) diskZone() maxShares(0)
I0929 15:36:51.655522       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029 StorageAccountType:StandardSSD_LRS Size:10
I0929 15:36:54.193514       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029 StorageAccountType:StandardSSD_LRS Size:10
I0929 15:36:54.193585       1 controllerserver.go:346] create azure disk(pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029 kubernetes.io-created-for-pvc-name:nginx-azuredisk-ephemeral-azuredisk01 kubernetes.io-created-for-pvc-namespace:default]) successfully
I0929 15:36:54.193607       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029","csi.storage.k8s.io/pvc/name":"nginx-azuredisk-ephemeral-azuredisk01","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuname":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029"}}
I0929 15:36:54.764195       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 2 lines ...
I0929 15:36:54.793709       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:36:54.793731       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:36:54.793766       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:36:54.793785       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029" lun 1 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029  true false 1})]
I0929 15:36:54.793828       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:36:54.793843       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029  true false 1})])
I0929 15:36:54.957644       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:37:05.093095       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029" attached to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:37:05.093151       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-dc057cb6-5998-42b3-9ccb-d60024ab4029" to node "k8s-agentpool1-18436497-vmss000001" successfully
I0929 15:37:05.093187       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
I0929 15:37:15.172672       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:37:15.172699       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-10d18411-10ec-46e1-b471-36210728c43c","parameters":{"csi.storage.k8s.io/pv/name":"pvc-10d18411-10ec-46e1-b471-36210728c43c","csi.storage.k8s.io/pvc/name":"daemonset-azuredisk-ephemeral-p4sh2-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","skuname":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:37:15.179491       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I0929 15:37:15.179514       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-451345f9-665f-4a90-84cb-e4dbabbb585f","parameters":{"csi.storage.k8s.io/pv/name":"pvc-451345f9-665f-4a90-84cb-e4dbabbb585f","csi.storage.k8s.io/pvc/name":"daemonset-azuredisk-ephemeral-2zslb-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","skuname":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I0929 15:37:15.192049       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-10d18411-10ec-46e1-b471-36210728c43c, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-10d18411-10ec-46e1-b471-36210728c43c' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:37:15.192118       1 controllerserver.go:269] begin to create azure disk(pvc-10d18411-10ec-46e1-b471-36210728c43c) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(10) diskZone() maxShares(0)
I0929 15:37:15.192152       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-10d18411-10ec-46e1-b471-36210728c43c StorageAccountType:StandardSSD_LRS Size:10
I0929 15:37:15.211424       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-451345f9-665f-4a90-84cb-e4dbabbb585f, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-451345f9-665f-4a90-84cb-e4dbabbb585f' under resource group 'kubetest-xhnqwtqz' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I0929 15:37:15.211506       1 controllerserver.go:269] begin to create azure disk(pvc-451345f9-665f-4a90-84cb-e4dbabbb585f) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(10) diskZone() maxShares(0)
I0929 15:37:15.211526       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-451345f9-665f-4a90-84cb-e4dbabbb585f StorageAccountType:StandardSSD_LRS Size:10
I0929 15:37:17.606163       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-10d18411-10ec-46e1-b471-36210728c43c StorageAccountType:StandardSSD_LRS Size:10
I0929 15:37:17.606203       1 controllerserver.go:346] create azure disk(pvc-10d18411-10ec-46e1-b471-36210728c43c) account type(StandardSSD_LRS) rg(kubetest-xhnqwtqz) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-10d18411-10ec-46e1-b471-36210728c43c kubernetes.io-created-for-pvc-name:daemonset-azuredisk-ephemeral-p4sh2-azuredisk kubernetes.io-created-for-pvc-namespace:default]) successfully
I0929 15:37:17.606245       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-10d18411-10ec-46e1-b471-36210728c43c","csi.storage.k8s.io/pvc/name":"daemonset-azuredisk-ephemeral-p4sh2-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuname":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-10d18411-10ec-46e1-b471-36210728c43c"}}
I0929 15:37:17.747353       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-451345f9-665f-4a90-84cb-e4dbabbb585f StorageAccountType:StandardSSD_LRS Size:10
... skipping 5 lines ...
I0929 15:37:18.879635       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:37:18.879679       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-10d18411-10ec-46e1-b471-36210728c43c" to node "k8s-agentpool1-18436497-vmss000000"
I0929 15:37:18.879718       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:37:18.879748       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-10d18411-10ec-46e1-b471-36210728c43c" lun 2 to node "k8s-agentpool1-18436497-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-10d18411-10ec-46e1-b471-36210728c43c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-10d18411-10ec-46e1-b471-36210728c43c  true false 2})]
I0929 15:37:18.879771       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:37:18.879805       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-10d18411-10ec-46e1-b471-36210728c43c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-10d18411-10ec-46e1-b471-36210728c43c  true false 2})])
I0929 15:37:19.647693       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-10d18411-10ec-46e1-b471-36210728c43c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-10d18411-10ec-46e1-b471-36210728c43c  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:37:19.858650       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:37:19.858680       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-451345f9-665f-4a90-84cb-e4dbabbb585f","csi.storage.k8s.io/pvc/name":"daemonset-azuredisk-ephemeral-2zslb-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuname":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-451345f9-665f-4a90-84cb-e4dbabbb585f"}
I0929 15:37:19.881077       1 controllerserver.go:438] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-451345f9-665f-4a90-84cb-e4dbabbb585f" to node "k8s-agentpool1-18436497-vmss000001".
I0929 15:37:19.881137       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:37:19.881173       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-18436497-vmss000001, refreshing the cache
I0929 15:37:19.975492       1 controllerserver.go:459] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-451345f9-665f-4a90-84cb-e4dbabbb585f" to node "k8s-agentpool1-18436497-vmss000001"
I0929 15:37:19.975554       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:37:19.975600       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-451345f9-665f-4a90-84cb-e4dbabbb585f" lun 2 to node "k8s-agentpool1-18436497-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-451345f9-665f-4a90-84cb-e4dbabbb585f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-451345f9-665f-4a90-84cb-e4dbabbb585f  true false 2})]
I0929 15:37:19.975635       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-18436497-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I0929 15:37:19.975665       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-451345f9-665f-4a90-84cb-e4dbabbb585f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-451345f9-665f-4a90-84cb-e4dbabbb585f  true false 2})])
I0929 15:37:20.184157       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xhnqwtqz): vm(k8s-agentpool1-18436497-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xhnqwtqz/providers/microsoft.compute/disks/pvc-451345f9-665f-4a90-84cb-e4dbabbb585f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-451345f9-665f-4a90-84cb-e4dbabbb585f  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I0929 15:37:20.512731       1 azure_vmss.go:319] Couldn't find VMSS VM with scaleSetName "k8s-agentpool1-18436497-vmss" and instanceID "1", refreshing the cache
I0929 15:37:34.859181       1 controllerserver.go:463] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-10d18411-10ec-46e1-b471-36210728c43c" attached to node "k8s-agentpool1-18436497-vmss000000".
I0929 15:37:34.859224       1 controllerserver.go:483] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-10d18411-10ec-46e1-b471-36210728c43c" to node "k8s-agentpool1-18436497-vmss000000" successfully
I0929 15:37:34.859264       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"2"}}
I0929 15:37:34.868976       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0929 15:37:34.869006       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-18436497-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-10d18411-10ec-46e1-b471-36210728c43c","csi.storage.k8s.io/pvc/name":"daemonset-azuredisk-ephemeral-p4sh2-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuname":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-10d18411-10ec-46e1-b471-36210728c43c"}
... skipping 28 lines ...
Platform: linux/amd64
Topology Key: topology.test.csi.azure.com/zone

Streaming logs below:
I0929 15:13:10.377164       1 azuredisk.go:136] driver userAgent: test.csi.azure.com/v1.8.0-48d7bd18383c09b02fba2b10739a91587fbfde8c gc/go1.17.1 (amd64-linux) e2e-test
I0929 15:13:10.378368       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W0929 15:13:10.408502       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I0929 15:13:10.408534       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I0929 15:13:10.408542       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I0929 15:13:10.408575       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I0929 15:13:10.409653       1 azure_auth.go:234] Using AzurePublicCloud environment
I0929 15:13:10.409674       1 azure_auth.go:96] azure: using managed identity extension to retrieve access token
I0929 15:13:10.409680       1 azure_auth.go:102] azure: using User Assigned MSI ID to retrieve access token
... skipping 867 lines ...
Platform: linux/amd64
Topology Key: topology.test.csi.azure.com/zone

Streaming logs below:
I0929 15:13:04.343456       1 azuredisk.go:136] driver userAgent: test.csi.azure.com/v1.8.0-48d7bd18383c09b02fba2b10739a91587fbfde8c gc/go1.17.1 (amd64-linux) e2e-test
I0929 15:13:04.346936       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W0929 15:13:04.368325       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I0929 15:13:04.368357       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I0929 15:13:04.368365       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I0929 15:13:04.368401       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I0929 15:13:04.375091       1 azure_auth.go:234] Using AzurePublicCloud environment
I0929 15:13:04.438154       1 azure_auth.go:96] azure: using managed identity extension to retrieve access token
I0929 15:13:04.438437       1 azure_auth.go:102] azure: using User Assigned MSI ID to retrieve access token
... skipping 72 lines ...
Platform: linux/amd64
Topology Key: topology.test.csi.azure.com/zone

Streaming logs below:
I0929 15:13:11.240538       1 azuredisk.go:136] driver userAgent: test.csi.azure.com/v1.8.0-48d7bd18383c09b02fba2b10739a91587fbfde8c gc/go1.17.1 (amd64-linux) e2e-test
I0929 15:13:11.241678       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W0929 15:13:11.262803       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I0929 15:13:11.262851       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I0929 15:13:11.262875       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I0929 15:13:11.262894       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I0929 15:13:11.263577       1 azure_auth.go:234] Using AzurePublicCloud environment
I0929 15:13:11.263598       1 azure_auth.go:96] azure: using managed identity extension to retrieve access token
I0929 15:13:11.263603       1 azure_auth.go:102] azure: using User Assigned MSI ID to retrieve access token
... skipping 939 lines ...
I0929 15:21:35.267209       1 utils.go:96] GRPC request: {"publish_context":{"LUN":"27"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6/globalmount","target_path":"/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6/mount","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6","csi.storage.k8s.io/pvc/name":"pvc-nm8dz","csi.storage.k8s.io/pvc/namespace":"volumelimits-8887","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1632928396898-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6"}
I0929 15:21:35.267381       1 nodeserver.go:261] NodePublishVolume: mounting /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6/globalmount at /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6/mount
I0929 15:21:35.267404       1 mount_linux.go:175] Mounting cmd (mount) with arguments ( -o bind /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6/globalmount /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6/mount)
I0929 15:21:35.269100       1 mount_linux.go:175] Mounting cmd (mount) with arguments ( -o bind,remount /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6/globalmount /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6/mount)
I0929 15:21:35.270624       1 nodeserver.go:266] NodePublishVolume: mount /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6/globalmount at /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6/mount successfully
I0929 15:21:35.270652       1 utils.go:102] GRPC response: {}
W0929 15:21:35.488293       1 azure_common_linux.go:180] azureDisk - getDiskLinkByDevName by sdad under /dev/disk/azure/scsi1/ failed, error: device name(sdad) is not found under /dev/disk/azure/scsi1/
W0929 15:21:35.488705       1 azure_common_linux.go:180] azureDisk - getDiskLinkByDevName by sdad under /dev/disk/by-id/ failed, error: device name(sdad) is not found under /dev/disk/by-id/
I0929 15:21:35.488737       1 nodeserver.go:110] NodeStageVolume: perf optimization is disabled for /dev/sdad. perfProfile none accountType 
I0929 15:21:35.488765       1 nodeserver.go:151] NodeStageVolume: formatting /dev/sdad and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e/globalmount with mount options([])
I0929 15:21:35.488777       1 mount_linux.go:446] Attempting to determine if disk "/dev/sdad" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/sdad])
I0929 15:21:35.493902       1 mount_linux.go:449] Output: ""
I0929 15:21:35.493933       1 mount_linux.go:408] Disk "/dev/sdad" appears to be unformatted, attempting to format as type: "ext4" with options: [-F -m0 /dev/sdad]
I0929 15:21:35.967096       1 mount_linux.go:418] Disk successfully formatted (mkfs): ext4 - /dev/sdad /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e/globalmount
... skipping 353 lines ...
I0929 15:25:14.226232       1 mount_helper_common.go:99] "/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6/mount" is a mountpoint, unmounting
I0929 15:25:14.226240       1 mount_linux.go:266] Unmounting /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6/mount
I0929 15:25:14.225889       1 utils.go:96] GRPC request: {"target_path":"/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e/mount","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e"}
I0929 15:25:14.226512       1 nodeserver.go:283] NodeUnpublishVolume: unmounting volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e on /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e/mount
I0929 15:25:14.226608       1 mount_helper_common.go:99] "/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e/mount" is a mountpoint, unmounting
I0929 15:25:14.226626       1 mount_linux.go:266] Unmounting /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-fb2fd193-8c5a-454b-9501-d37ee193c96e/mount
E0929 15:25:14.236491       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to unmount target "/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-432c9e62-39b8-44ac-bf04-83e34b649318/mount": could not get consistent content of /proc/mounts after 3 attempts
I0929 15:25:14.237887       1 utils.go:95] GRPC call: /csi.v1.Node/NodeUnpublishVolume
I0929 15:25:14.237911       1 utils.go:96] GRPC request: {"target_path":"/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-ed215498-867b-4073-8a88-6dbf534db251/mount","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251"}
I0929 15:25:14.237959       1 nodeserver.go:283] NodeUnpublishVolume: unmounting volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251 on /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-ed215498-867b-4073-8a88-6dbf534db251/mount
I0929 15:25:14.238002       1 mount_helper_common.go:99] "/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-ed215498-867b-4073-8a88-6dbf534db251/mount" is a mountpoint, unmounting
I0929 15:25:14.238019       1 mount_linux.go:266] Unmounting /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-ed215498-867b-4073-8a88-6dbf534db251/mount
I0929 15:25:14.238034       1 utils.go:95] GRPC call: /csi.v1.Node/NodeUnpublishVolume
... skipping 57 lines ...
I0929 15:25:14.335234       1 utils.go:102] GRPC response: {}
I0929 15:25:14.335383       1 utils.go:95] GRPC call: /csi.v1.Node/NodeUnpublishVolume
I0929 15:25:14.335392       1 utils.go:96] GRPC request: {"target_path":"/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030/mount","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030"}
I0929 15:25:14.335425       1 nodeserver.go:283] NodeUnpublishVolume: unmounting volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030 on /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030/mount
I0929 15:25:14.335451       1 mount_helper_common.go:99] "/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030/mount" is a mountpoint, unmounting
I0929 15:25:14.335458       1 mount_linux.go:266] Unmounting /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-8818b44e-c5c9-43ca-bf04-8b3c474d3030/mount
E0929 15:25:14.335862       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to unmount target "/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-96a41008-4f9b-416f-a3ab-21561bf55cb4/mount": could not get consistent content of /proc/mounts after 3 attempts
I0929 15:25:14.335921       1 utils.go:95] GRPC call: /csi.v1.Node/NodeUnpublishVolume
I0929 15:25:14.335972       1 utils.go:96] GRPC request: {"target_path":"/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b/mount","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b"}
I0929 15:25:14.336004       1 nodeserver.go:283] NodeUnpublishVolume: unmounting volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b on /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b/mount
I0929 15:25:14.336035       1 mount_helper_common.go:99] "/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b/mount" is a mountpoint, unmounting
I0929 15:25:14.336042       1 mount_linux.go:266] Unmounting /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-ed69c59c-f406-4e8a-b363-07f43ef4913b/mount
I0929 15:25:14.336394       1 utils.go:95] GRPC call: /csi.v1.Node/NodeUnpublishVolume
... skipping 91 lines ...
I0929 15:25:14.636796       1 utils.go:102] GRPC response: {}
W0929 15:25:14.636812       1 mount_helper_common.go:129] Warning: "/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e/mount" is not a mountpoint, deleting
I0929 15:25:14.636824       1 nodeserver.go:289] NodeUnpublishVolume: unmount volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ed215498-867b-4073-8a88-6dbf534db251 on /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-ed215498-867b-4073-8a88-6dbf534db251/mount successfully
I0929 15:25:14.636833       1 utils.go:102] GRPC response: {}
I0929 15:25:14.636865       1 nodeserver.go:289] NodeUnpublishVolume: unmount volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e on /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-1a02bdf7-5624-43f2-ba01-2a7d3c0f2e1e/mount successfully
I0929 15:25:14.636874       1 utils.go:102] GRPC response: {}
E0929 15:25:14.637162       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to unmount target "/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-947490b0-7614-49a1-a791-0edb87f913dd/mount": could not get consistent content of /proc/mounts after 3 attempts
W0929 15:25:14.637288       1 mount_helper_common.go:129] Warning: "/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6/mount" is not a mountpoint, deleting
W0929 15:25:14.637321       1 mount_helper_common.go:129] Warning: "/var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e/mount" is not a mountpoint, deleting
I0929 15:25:14.637348       1 nodeserver.go:289] NodeUnpublishVolume: unmount volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6 on /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-7e63d85d-4075-4178-8ab3-ef2f42c61de6/mount successfully
I0929 15:25:14.637358       1 utils.go:102] GRPC response: {}
I0929 15:25:14.637375       1 nodeserver.go:289] NodeUnpublishVolume: unmount volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xhnqwtqz/providers/Microsoft.Compute/disks/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e on /var/lib/kubelet/pods/e3750775-1de9-4022-b40d-b26b8d32bea8/volumes/kubernetes.io~csi/pvc-ca999c3b-0fab-45ef-8ac0-7725719aa12e/mount successfully
I0929 15:25:14.637390       1 utils.go:102] GRPC response: {}
... skipping 640 lines ...
2021/09/29 15:38:57 process.go:153: Running: chmod +x /root/tmp2380974403/win-ci-logs-collector.sh
2021/09/29 15:38:57 process.go:155: Step 'chmod +x /root/tmp2380974403/win-ci-logs-collector.sh' finished in 1.405995ms
2021/09/29 15:38:57 process.go:153: Running: bash -c /root/tmp2380974403/win-ci-logs-collector.sh kubetest-xhnqwtqz.westeurope.cloudapp.azure.com /root/tmp2380974403 /root/.ssh/id_rsa
ssh key file /root/.ssh/id_rsa does not exist. Exiting.
2021/09/29 15:38:57 process.go:155: Step 'bash -c /root/tmp2380974403/win-ci-logs-collector.sh kubetest-xhnqwtqz.westeurope.cloudapp.azure.com /root/tmp2380974403 /root/.ssh/id_rsa' finished in 4.048753ms
2021/09/29 15:38:57 aksengine.go:1141: Deleting resource group: kubetest-xhnqwtqz.
{"component":"entrypoint","file":"prow/entrypoint/run.go:169","func":"k8s.io/test-infra/prow/entrypoint.Options.ExecuteProcess","level":"error","msg":"Entrypoint received interrupt: terminated","severity":"error","time":"2021-09-29T15:46:04Z"}
++ early_exit_handler
++ '[' -n 178 ']'
++ kill -TERM 178
++ cleanup_dind
++ [[ true == \t\r\u\e ]]
++ echo 'Cleaning up after docker'
... skipping 4 lines ...