This job view page is being replaced by Spyglass soon. Check out the new job view.
PRandyzhangx: fix: get MaxDataDiskCount error on Windows
ResultFAILURE
Tests 1 failed / 11 succeeded
Started2021-10-02 14:32
Elapsed53m44s
Revisionb8ffb8bbe543dff095894b940a9f6dccff23a82a
Refs 1039
job-versionv1.23.0-alpha.3.105+0ac956ff2bef9c
kubetest-version
revisionv1.23.0-alpha.3.105+0ac956ff2bef9c

Test Failures


kubetest Test 39m50s

error during make e2e-test: exit status 2
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 11 Passed Tests

Error lines from build-log.txt

... skipping 222 lines ...

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 11248  100 11248    0     0   180k      0 --:--:-- --:--:-- --:--:--  183k
Downloading https://get.helm.sh/helm-v3.7.0-linux-amd64.tar.gz
Verifying checksum... Done.
Preparing to install helm into /usr/local/bin
helm installed into /usr/local/bin/helm
docker pull k8sprow.azurecr.io/azuredisk-csi:v1.8.0-b1f99a5a1f102374a7a848cd09676c7bf2753657 || make container-all push-manifest
Error response from daemon: manifest for k8sprow.azurecr.io/azuredisk-csi:v1.8.0-b1f99a5a1f102374a7a848cd09676c7bf2753657 not found: manifest unknown: manifest tagged by "v1.8.0-b1f99a5a1f102374a7a848cd09676c7bf2753657" is not found
make[2]: Entering directory '/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver'
CGO_ENABLED=0 GOOS=windows go build -a -ldflags "-X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.driverVersion=v1.8.0-b1f99a5a1f102374a7a848cd09676c7bf2753657 -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.gitCommit=b1f99a5a1f102374a7a848cd09676c7bf2753657 -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.buildDate=2021-10-02T14:37:57Z -extldflags "-static""  -mod vendor -o _output/amd64/azurediskplugin.exe ./pkg/azurediskplugin
docker buildx rm container-builder || true
error: no builder "container-builder" found
docker buildx create --use --name=container-builder
container-builder
# enable qemu for arm64 build
# https://github.com/docker/buildx/issues/464#issuecomment-741507760
docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-aarch64
Unable to find image 'tonistiigi/binfmt:latest' locally
... skipping 1787 lines ...
                    type: string
                type: object
                oneOf:
                - required: ["persistentVolumeClaimName"]
                - required: ["volumeSnapshotContentName"]
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 60 lines ...
                    type: string
                  volumeSnapshotContentName:
                    description: volumeSnapshotContentName specifies the name of a pre-existing VolumeSnapshotContent object representing an existing volume snapshot. This field should be set if the snapshot already exists and only needs a representation in Kubernetes. This field is immutable.
                    type: string
                type: object
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 248 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 108 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 850 lines ...
        - name: csi-resizer
          image: "mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0"
          args:
            - "-csi-address=$(ADDRESS)"
            - "-v=2"
            - "-leader-election"
            - '-handle-volume-inuse-error=true'
          env:
            - name: ADDRESS
              value: /csi/csi.sock
          volumeMounts:
            - name: socket-dir
              mountPath: /csi
... skipping 523 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (filesystem volmode)] volumeMode
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should not mount / map unused volumes in a pod [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:351
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode should not mount / map unused volumes in a pod [LinuxOnly]","total":20,"completed":1,"skipped":39,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 14:47:21.484: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 40 lines ...
Oct  2 14:46:17.394: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com4vvb9] to have phase Bound
Oct  2 14:46:17.503: INFO: PersistentVolumeClaim test.csi.azure.com4vvb9 found but phase is Pending instead of Bound.
Oct  2 14:46:19.615: INFO: PersistentVolumeClaim test.csi.azure.com4vvb9 found but phase is Pending instead of Bound.
Oct  2 14:46:21.724: INFO: PersistentVolumeClaim test.csi.azure.com4vvb9 found and phase=Bound (4.330225388s)
STEP: Creating pod pod-subpath-test-dynamicpv-gfhn
STEP: Creating a pod to test subpath
Oct  2 14:46:22.082: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-gfhn" in namespace "provisioning-4498" to be "Succeeded or Failed"
Oct  2 14:46:22.195: INFO: Pod "pod-subpath-test-dynamicpv-gfhn": Phase="Pending", Reason="", readiness=false. Elapsed: 112.196235ms
Oct  2 14:46:24.304: INFO: Pod "pod-subpath-test-dynamicpv-gfhn": Phase="Pending", Reason="", readiness=false. Elapsed: 2.221975203s
Oct  2 14:46:26.415: INFO: Pod "pod-subpath-test-dynamicpv-gfhn": Phase="Pending", Reason="", readiness=false. Elapsed: 4.332697814s
Oct  2 14:46:28.526: INFO: Pod "pod-subpath-test-dynamicpv-gfhn": Phase="Pending", Reason="", readiness=false. Elapsed: 6.443403235s
Oct  2 14:46:30.638: INFO: Pod "pod-subpath-test-dynamicpv-gfhn": Phase="Pending", Reason="", readiness=false. Elapsed: 8.555504814s
Oct  2 14:46:32.748: INFO: Pod "pod-subpath-test-dynamicpv-gfhn": Phase="Pending", Reason="", readiness=false. Elapsed: 10.66550118s
... skipping 9 lines ...
Oct  2 14:46:53.866: INFO: Pod "pod-subpath-test-dynamicpv-gfhn": Phase="Pending", Reason="", readiness=false. Elapsed: 31.783575987s
Oct  2 14:46:55.976: INFO: Pod "pod-subpath-test-dynamicpv-gfhn": Phase="Pending", Reason="", readiness=false. Elapsed: 33.893257381s
Oct  2 14:46:58.094: INFO: Pod "pod-subpath-test-dynamicpv-gfhn": Phase="Pending", Reason="", readiness=false. Elapsed: 36.011425893s
Oct  2 14:47:00.204: INFO: Pod "pod-subpath-test-dynamicpv-gfhn": Phase="Pending", Reason="", readiness=false. Elapsed: 38.121928282s
Oct  2 14:47:02.316: INFO: Pod "pod-subpath-test-dynamicpv-gfhn": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.233995245s
STEP: Saw pod success
Oct  2 14:47:02.317: INFO: Pod "pod-subpath-test-dynamicpv-gfhn" satisfied condition "Succeeded or Failed"
Oct  2 14:47:02.426: INFO: Trying to get logs from node k8s-agentpool1-13787793-vmss000001 pod pod-subpath-test-dynamicpv-gfhn container test-container-volume-dynamicpv-gfhn: <nil>
STEP: delete the pod
Oct  2 14:47:02.688: INFO: Waiting for pod pod-subpath-test-dynamicpv-gfhn to disappear
Oct  2 14:47:02.797: INFO: Pod pod-subpath-test-dynamicpv-gfhn no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-gfhn
Oct  2 14:47:02.797: INFO: Deleting pod "pod-subpath-test-dynamicpv-gfhn" in namespace "provisioning-4498"
... skipping 23 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support existing directory
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:205
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directory","total":17,"completed":1,"skipped":126,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes 
  should allow exec of files on the volume
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:196
... skipping 19 lines ...
Oct  2 14:46:17.498: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com6x7zh] to have phase Bound
Oct  2 14:46:17.606: INFO: PersistentVolumeClaim test.csi.azure.com6x7zh found but phase is Pending instead of Bound.
Oct  2 14:46:19.715: INFO: PersistentVolumeClaim test.csi.azure.com6x7zh found but phase is Pending instead of Bound.
Oct  2 14:46:21.825: INFO: PersistentVolumeClaim test.csi.azure.com6x7zh found and phase=Bound (4.327310357s)
STEP: Creating pod exec-volume-test-dynamicpv-dlmc
STEP: Creating a pod to test exec-volume-test
Oct  2 14:46:22.160: INFO: Waiting up to 5m0s for pod "exec-volume-test-dynamicpv-dlmc" in namespace "volume-5831" to be "Succeeded or Failed"
Oct  2 14:46:22.268: INFO: Pod "exec-volume-test-dynamicpv-dlmc": Phase="Pending", Reason="", readiness=false. Elapsed: 107.784724ms
Oct  2 14:46:24.378: INFO: Pod "exec-volume-test-dynamicpv-dlmc": Phase="Pending", Reason="", readiness=false. Elapsed: 2.218080953s
Oct  2 14:46:26.488: INFO: Pod "exec-volume-test-dynamicpv-dlmc": Phase="Pending", Reason="", readiness=false. Elapsed: 4.327685493s
Oct  2 14:46:28.599: INFO: Pod "exec-volume-test-dynamicpv-dlmc": Phase="Pending", Reason="", readiness=false. Elapsed: 6.438332513s
Oct  2 14:46:30.709: INFO: Pod "exec-volume-test-dynamicpv-dlmc": Phase="Pending", Reason="", readiness=false. Elapsed: 8.548670833s
Oct  2 14:46:32.818: INFO: Pod "exec-volume-test-dynamicpv-dlmc": Phase="Pending", Reason="", readiness=false. Elapsed: 10.6578905s
... skipping 10 lines ...
Oct  2 14:46:56.029: INFO: Pod "exec-volume-test-dynamicpv-dlmc": Phase="Pending", Reason="", readiness=false. Elapsed: 33.86848501s
Oct  2 14:46:58.138: INFO: Pod "exec-volume-test-dynamicpv-dlmc": Phase="Pending", Reason="", readiness=false. Elapsed: 35.977701311s
Oct  2 14:47:00.248: INFO: Pod "exec-volume-test-dynamicpv-dlmc": Phase="Pending", Reason="", readiness=false. Elapsed: 38.08812139s
Oct  2 14:47:02.358: INFO: Pod "exec-volume-test-dynamicpv-dlmc": Phase="Pending", Reason="", readiness=false. Elapsed: 40.197235873s
Oct  2 14:47:04.472: INFO: Pod "exec-volume-test-dynamicpv-dlmc": Phase="Succeeded", Reason="", readiness=false. Elapsed: 42.312122766s
STEP: Saw pod success
Oct  2 14:47:04.472: INFO: Pod "exec-volume-test-dynamicpv-dlmc" satisfied condition "Succeeded or Failed"
Oct  2 14:47:04.581: INFO: Trying to get logs from node k8s-agentpool1-13787793-vmss000000 pod exec-volume-test-dynamicpv-dlmc container exec-container-dynamicpv-dlmc: <nil>
STEP: delete the pod
Oct  2 14:47:04.810: INFO: Waiting for pod exec-volume-test-dynamicpv-dlmc to disappear
Oct  2 14:47:04.918: INFO: Pod exec-volume-test-dynamicpv-dlmc no longer exists
STEP: Deleting pod exec-volume-test-dynamicpv-dlmc
Oct  2 14:47:04.918: INFO: Deleting pod "exec-volume-test-dynamicpv-dlmc" in namespace "volume-5831"
... skipping 21 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should allow exec of files on the volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:196
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should allow exec of files on the volume","total":14,"completed":1,"skipped":207,"failed":0}

SSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 14:47:46.636: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 117 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","total":15,"completed":1,"skipped":261,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (delayed binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (delayed binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Oct  2 14:48:22.360: INFO: >>> kubeConfig: /root/tmp2688293530/kubeconfig/kubeconfig.westeurope.json
STEP: Building a namespace api object, basename topology
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail to schedule a pod which has topologies that conflict with AllowedTopologies
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192
Oct  2 14:48:22.897: INFO: Driver didn't provide topology keys -- skipping
[AfterEach] [Testpattern: Dynamic PV (delayed binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Oct  2 14:48:22.898: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "topology-8049" for this suite.


S [SKIPPING] [0.761 seconds]
External Storage [Driver: test.csi.azure.com]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (delayed binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to schedule a pod which has topologies that conflict with AllowedTopologies [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192

    Driver didn't provide topology keys -- skipping

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:124
------------------------------
... skipping 205 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read/write inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:166
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read/write inline ephemeral volume","total":16,"completed":1,"skipped":56,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 14:48:23.847: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 61 lines ...
Oct  2 14:46:17.284: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com5q9hx] to have phase Bound
Oct  2 14:46:17.392: INFO: PersistentVolumeClaim test.csi.azure.com5q9hx found but phase is Pending instead of Bound.
Oct  2 14:46:19.499: INFO: PersistentVolumeClaim test.csi.azure.com5q9hx found but phase is Pending instead of Bound.
Oct  2 14:46:21.607: INFO: PersistentVolumeClaim test.csi.azure.com5q9hx found and phase=Bound (4.322451747s)
STEP: Creating pod pod-subpath-test-dynamicpv-5tvl
STEP: Creating a pod to test atomic-volume-subpath
Oct  2 14:46:21.936: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-5tvl" in namespace "provisioning-87" to be "Succeeded or Failed"
Oct  2 14:46:22.045: INFO: Pod "pod-subpath-test-dynamicpv-5tvl": Phase="Pending", Reason="", readiness=false. Elapsed: 109.262097ms
Oct  2 14:46:24.154: INFO: Pod "pod-subpath-test-dynamicpv-5tvl": Phase="Pending", Reason="", readiness=false. Elapsed: 2.218687644s
Oct  2 14:46:26.263: INFO: Pod "pod-subpath-test-dynamicpv-5tvl": Phase="Pending", Reason="", readiness=false. Elapsed: 4.326899896s
Oct  2 14:46:28.372: INFO: Pod "pod-subpath-test-dynamicpv-5tvl": Phase="Pending", Reason="", readiness=false. Elapsed: 6.436053635s
Oct  2 14:46:30.481: INFO: Pod "pod-subpath-test-dynamicpv-5tvl": Phase="Pending", Reason="", readiness=false. Elapsed: 8.545221845s
Oct  2 14:46:32.589: INFO: Pod "pod-subpath-test-dynamicpv-5tvl": Phase="Pending", Reason="", readiness=false. Elapsed: 10.652920433s
... skipping 19 lines ...
Oct  2 14:47:14.793: INFO: Pod "pod-subpath-test-dynamicpv-5tvl": Phase="Running", Reason="", readiness=true. Elapsed: 52.857797983s
Oct  2 14:47:16.901: INFO: Pod "pod-subpath-test-dynamicpv-5tvl": Phase="Running", Reason="", readiness=true. Elapsed: 54.965148212s
Oct  2 14:47:19.017: INFO: Pod "pod-subpath-test-dynamicpv-5tvl": Phase="Running", Reason="", readiness=true. Elapsed: 57.080955905s
Oct  2 14:47:21.128: INFO: Pod "pod-subpath-test-dynamicpv-5tvl": Phase="Running", Reason="", readiness=true. Elapsed: 59.192114326s
Oct  2 14:47:23.236: INFO: Pod "pod-subpath-test-dynamicpv-5tvl": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m1.300191758s
STEP: Saw pod success
Oct  2 14:47:23.236: INFO: Pod "pod-subpath-test-dynamicpv-5tvl" satisfied condition "Succeeded or Failed"
Oct  2 14:47:23.346: INFO: Trying to get logs from node k8s-agentpool1-13787793-vmss000000 pod pod-subpath-test-dynamicpv-5tvl container test-container-subpath-dynamicpv-5tvl: <nil>
STEP: delete the pod
Oct  2 14:47:23.574: INFO: Waiting for pod pod-subpath-test-dynamicpv-5tvl to disappear
Oct  2 14:47:23.681: INFO: Pod pod-subpath-test-dynamicpv-5tvl no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-5tvl
Oct  2 14:47:23.681: INFO: Deleting pod "pod-subpath-test-dynamicpv-5tvl" in namespace "provisioning-87"
... skipping 29 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support file as subpath [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:230
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support file as subpath [LinuxOnly]","total":13,"completed":1,"skipped":59,"failed":0}

SSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (delayed binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (delayed binding)] topology
... skipping 209 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)] volumeMode
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should not mount / map unused volumes in a pod [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:351
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] volumeMode should not mount / map unused volumes in a pod [LinuxOnly]","total":20,"completed":2,"skipped":164,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral 
  should create read-only inline ephemeral volume
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149
... skipping 43 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read-only inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read-only inline ephemeral volume","total":14,"completed":2,"skipped":425,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 14:49:14.821: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 73 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support multiple inline ephemeral volumes
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:221
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support multiple inline ephemeral volumes","total":17,"completed":2,"skipped":153,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 14:49:55.805: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 98 lines ...
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Oct  2 14:49:57.288: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volumelimits-2975" for this suite.

•
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits should verify that all csinodes have volume limits","total":17,"completed":3,"skipped":378,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] provisioning
... skipping 104 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should store data
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] volumes should store data","total":20,"completed":3,"skipped":192,"failed":0}

SSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: CSI Ephemeral-volume (default fs)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 14:50:34.063: INFO: Driver "test.csi.azure.com" does not support volume type "CSIInlineVolume" - skipping
... skipping 121 lines ...
Oct  2 14:48:38.978: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comgx49z] to have phase Bound
Oct  2 14:48:39.090: INFO: PersistentVolumeClaim test.csi.azure.comgx49z found but phase is Pending instead of Bound.
Oct  2 14:48:41.198: INFO: PersistentVolumeClaim test.csi.azure.comgx49z found but phase is Pending instead of Bound.
Oct  2 14:48:43.307: INFO: PersistentVolumeClaim test.csi.azure.comgx49z found and phase=Bound (4.329247111s)
STEP: Creating pod pod-subpath-test-dynamicpv-np8f
STEP: Creating a pod to test subpath
Oct  2 14:48:43.634: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-np8f" in namespace "provisioning-25" to be "Succeeded or Failed"
Oct  2 14:48:43.740: INFO: Pod "pod-subpath-test-dynamicpv-np8f": Phase="Pending", Reason="", readiness=false. Elapsed: 106.548064ms
Oct  2 14:48:45.859: INFO: Pod "pod-subpath-test-dynamicpv-np8f": Phase="Pending", Reason="", readiness=false. Elapsed: 2.224865769s
Oct  2 14:48:47.968: INFO: Pod "pod-subpath-test-dynamicpv-np8f": Phase="Pending", Reason="", readiness=false. Elapsed: 4.334193583s
Oct  2 14:48:50.076: INFO: Pod "pod-subpath-test-dynamicpv-np8f": Phase="Pending", Reason="", readiness=false. Elapsed: 6.442269008s
Oct  2 14:48:52.186: INFO: Pod "pod-subpath-test-dynamicpv-np8f": Phase="Pending", Reason="", readiness=false. Elapsed: 8.551825414s
Oct  2 14:48:54.300: INFO: Pod "pod-subpath-test-dynamicpv-np8f": Phase="Pending", Reason="", readiness=false. Elapsed: 10.666415618s
... skipping 23 lines ...
Oct  2 14:49:44.909: INFO: Pod "pod-subpath-test-dynamicpv-np8f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m1.274798998s
Oct  2 14:49:47.019: INFO: Pod "pod-subpath-test-dynamicpv-np8f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m3.385066294s
Oct  2 14:49:49.127: INFO: Pod "pod-subpath-test-dynamicpv-np8f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m5.493669531s
Oct  2 14:49:51.236: INFO: Pod "pod-subpath-test-dynamicpv-np8f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m7.601904379s
Oct  2 14:49:53.344: INFO: Pod "pod-subpath-test-dynamicpv-np8f": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m9.710426936s
STEP: Saw pod success
Oct  2 14:49:53.344: INFO: Pod "pod-subpath-test-dynamicpv-np8f" satisfied condition "Succeeded or Failed"
Oct  2 14:49:53.452: INFO: Trying to get logs from node k8s-agentpool1-13787793-vmss000001 pod pod-subpath-test-dynamicpv-np8f container test-container-volume-dynamicpv-np8f: <nil>
STEP: delete the pod
Oct  2 14:49:53.737: INFO: Waiting for pod pod-subpath-test-dynamicpv-np8f to disappear
Oct  2 14:49:53.845: INFO: Pod pod-subpath-test-dynamicpv-np8f no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-np8f
Oct  2 14:49:53.845: INFO: Deleting pod "pod-subpath-test-dynamicpv-np8f" in namespace "provisioning-25"
... skipping 29 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support non-existent path
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:194
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support non-existent path","total":13,"completed":2,"skipped":542,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 14:51:06.485: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 100 lines ...
Oct  2 14:49:15.680: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com2plmb] to have phase Bound
Oct  2 14:49:15.788: INFO: PersistentVolumeClaim test.csi.azure.com2plmb found but phase is Pending instead of Bound.
Oct  2 14:49:17.898: INFO: PersistentVolumeClaim test.csi.azure.com2plmb found but phase is Pending instead of Bound.
Oct  2 14:49:20.007: INFO: PersistentVolumeClaim test.csi.azure.com2plmb found and phase=Bound (4.327547347s)
STEP: Creating pod pod-subpath-test-dynamicpv-5242
STEP: Creating a pod to test subpath
Oct  2 14:49:20.337: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-5242" in namespace "provisioning-3432" to be "Succeeded or Failed"
Oct  2 14:49:20.445: INFO: Pod "pod-subpath-test-dynamicpv-5242": Phase="Pending", Reason="", readiness=false. Elapsed: 108.431554ms
Oct  2 14:49:22.556: INFO: Pod "pod-subpath-test-dynamicpv-5242": Phase="Pending", Reason="", readiness=false. Elapsed: 2.21904946s
Oct  2 14:49:24.665: INFO: Pod "pod-subpath-test-dynamicpv-5242": Phase="Pending", Reason="", readiness=false. Elapsed: 4.328414778s
Oct  2 14:49:26.776: INFO: Pod "pod-subpath-test-dynamicpv-5242": Phase="Pending", Reason="", readiness=false. Elapsed: 6.439085565s
Oct  2 14:49:28.885: INFO: Pod "pod-subpath-test-dynamicpv-5242": Phase="Pending", Reason="", readiness=false. Elapsed: 8.548538533s
Oct  2 14:49:30.996: INFO: Pod "pod-subpath-test-dynamicpv-5242": Phase="Pending", Reason="", readiness=false. Elapsed: 10.659447339s
... skipping 8 lines ...
Oct  2 14:49:49.994: INFO: Pod "pod-subpath-test-dynamicpv-5242": Phase="Pending", Reason="", readiness=false. Elapsed: 29.656954058s
Oct  2 14:49:52.104: INFO: Pod "pod-subpath-test-dynamicpv-5242": Phase="Pending", Reason="", readiness=false. Elapsed: 31.766788665s
Oct  2 14:49:54.212: INFO: Pod "pod-subpath-test-dynamicpv-5242": Phase="Pending", Reason="", readiness=false. Elapsed: 33.875617992s
Oct  2 14:49:56.330: INFO: Pod "pod-subpath-test-dynamicpv-5242": Phase="Pending", Reason="", readiness=false. Elapsed: 35.993224659s
Oct  2 14:49:58.446: INFO: Pod "pod-subpath-test-dynamicpv-5242": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.109068397s
STEP: Saw pod success
Oct  2 14:49:58.446: INFO: Pod "pod-subpath-test-dynamicpv-5242" satisfied condition "Succeeded or Failed"
Oct  2 14:49:58.554: INFO: Trying to get logs from node k8s-agentpool1-13787793-vmss000001 pod pod-subpath-test-dynamicpv-5242 container test-container-subpath-dynamicpv-5242: <nil>
STEP: delete the pod
Oct  2 14:49:58.785: INFO: Waiting for pod pod-subpath-test-dynamicpv-5242 to disappear
Oct  2 14:49:58.894: INFO: Pod pod-subpath-test-dynamicpv-5242 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-5242
Oct  2 14:49:58.894: INFO: Deleting pod "pod-subpath-test-dynamicpv-5242" in namespace "provisioning-3432"
... skipping 29 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support existing single file [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:219
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing single file [LinuxOnly]","total":14,"completed":3,"skipped":527,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (ext4)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 14:51:11.481: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 206 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support two pods which share the same volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:183
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which share the same volume","total":10,"completed":1,"skipped":239,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 14:51:13.989: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 113 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should be able to unmount after the subpath directory is deleted [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:445
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should be able to unmount after the subpath directory is deleted [LinuxOnly]","total":17,"completed":4,"skipped":548,"failed":0}

SSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)(allowExpansion)][Feature:Windows] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 14:51:59.645: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 111 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","total":20,"completed":4,"skipped":587,"failed":0}

S
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (immediate binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (immediate binding)] topology
... skipping 70 lines ...
[BeforeEach] [Testpattern: Dynamic PV (immediate binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Oct  2 14:53:05.231: INFO: >>> kubeConfig: /root/tmp2688293530/kubeconfig/kubeconfig.westeurope.json
STEP: Building a namespace api object, basename topology
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail to schedule a pod which has topologies that conflict with AllowedTopologies
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192
Oct  2 14:53:05.777: INFO: Driver didn't provide topology keys -- skipping
[AfterEach] [Testpattern: Dynamic PV (immediate binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Oct  2 14:53:05.777: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "topology-2313" for this suite.


S [SKIPPING] [0.786 seconds]
External Storage [Driver: test.csi.azure.com]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (immediate binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to schedule a pod which has topologies that conflict with AllowedTopologies [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192

    Driver didn't provide topology keys -- skipping

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:124
------------------------------
... skipping 81 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","total":13,"completed":3,"skipped":867,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
Oct  2 14:53:21.451: INFO: Running AfterSuite actions on all nodes
Oct  2 14:53:21.451: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func17.2
Oct  2 14:53:21.451: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func8.2
... skipping 86 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","total":17,"completed":5,"skipped":685,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (ntfs)][Feature:Windows] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 14:53:38.218: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 140 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:201
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]","total":10,"completed":2,"skipped":640,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
... skipping 163 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents","total":20,"completed":5,"skipped":733,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: CSI Ephemeral-volume (default fs)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 14:55:24.036: INFO: Driver "test.csi.azure.com" does not support volume type "CSIInlineVolume" - skipping
... skipping 64 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read/write inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:166
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read/write inline ephemeral volume","total":17,"completed":6,"skipped":895,"failed":0}

SSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 14:55:30.448: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 107 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should store data
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should store data","total":10,"completed":3,"skipped":842,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
Oct  2 14:56:47.968: INFO: Running AfterSuite actions on all nodes
Oct  2 14:56:47.969: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func17.2
Oct  2 14:56:47.969: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func8.2
... skipping 26 lines ...
Oct  2 14:55:24.817: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.compssbf] to have phase Bound
Oct  2 14:55:24.926: INFO: PersistentVolumeClaim test.csi.azure.compssbf found but phase is Pending instead of Bound.
Oct  2 14:55:27.035: INFO: PersistentVolumeClaim test.csi.azure.compssbf found but phase is Pending instead of Bound.
Oct  2 14:55:29.146: INFO: PersistentVolumeClaim test.csi.azure.compssbf found and phase=Bound (4.328163196s)
STEP: Creating pod pod-subpath-test-dynamicpv-f7cz
STEP: Creating a pod to test subpath
Oct  2 14:55:29.470: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-f7cz" in namespace "provisioning-1687" to be "Succeeded or Failed"
Oct  2 14:55:29.579: INFO: Pod "pod-subpath-test-dynamicpv-f7cz": Phase="Pending", Reason="", readiness=false. Elapsed: 108.165764ms
Oct  2 14:55:31.692: INFO: Pod "pod-subpath-test-dynamicpv-f7cz": Phase="Pending", Reason="", readiness=false. Elapsed: 2.221170017s
Oct  2 14:55:33.800: INFO: Pod "pod-subpath-test-dynamicpv-f7cz": Phase="Pending", Reason="", readiness=false. Elapsed: 4.329184975s
Oct  2 14:55:35.909: INFO: Pod "pod-subpath-test-dynamicpv-f7cz": Phase="Pending", Reason="", readiness=false. Elapsed: 6.438589922s
Oct  2 14:55:38.019: INFO: Pod "pod-subpath-test-dynamicpv-f7cz": Phase="Pending", Reason="", readiness=false. Elapsed: 8.548335403s
Oct  2 14:55:40.129: INFO: Pod "pod-subpath-test-dynamicpv-f7cz": Phase="Pending", Reason="", readiness=false. Elapsed: 10.658480247s
... skipping 8 lines ...
Oct  2 14:55:59.115: INFO: Pod "pod-subpath-test-dynamicpv-f7cz": Phase="Pending", Reason="", readiness=false. Elapsed: 29.644844819s
Oct  2 14:56:01.225: INFO: Pod "pod-subpath-test-dynamicpv-f7cz": Phase="Pending", Reason="", readiness=false. Elapsed: 31.754477219s
Oct  2 14:56:03.343: INFO: Pod "pod-subpath-test-dynamicpv-f7cz": Phase="Pending", Reason="", readiness=false. Elapsed: 33.872551671s
Oct  2 14:56:05.452: INFO: Pod "pod-subpath-test-dynamicpv-f7cz": Phase="Pending", Reason="", readiness=false. Elapsed: 35.981084016s
Oct  2 14:56:07.560: INFO: Pod "pod-subpath-test-dynamicpv-f7cz": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.089383873s
STEP: Saw pod success
Oct  2 14:56:07.560: INFO: Pod "pod-subpath-test-dynamicpv-f7cz" satisfied condition "Succeeded or Failed"
Oct  2 14:56:07.667: INFO: Trying to get logs from node k8s-agentpool1-13787793-vmss000001 pod pod-subpath-test-dynamicpv-f7cz container test-container-subpath-dynamicpv-f7cz: <nil>
STEP: delete the pod
Oct  2 14:56:07.938: INFO: Waiting for pod pod-subpath-test-dynamicpv-f7cz to disappear
Oct  2 14:56:08.046: INFO: Pod pod-subpath-test-dynamicpv-f7cz no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-f7cz
Oct  2 14:56:08.046: INFO: Deleting pod "pod-subpath-test-dynamicpv-f7cz" in namespace "provisioning-1687"
STEP: Creating pod pod-subpath-test-dynamicpv-f7cz
STEP: Creating a pod to test subpath
Oct  2 14:56:08.276: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-f7cz" in namespace "provisioning-1687" to be "Succeeded or Failed"
Oct  2 14:56:08.383: INFO: Pod "pod-subpath-test-dynamicpv-f7cz": Phase="Pending", Reason="", readiness=false. Elapsed: 106.838406ms
Oct  2 14:56:10.492: INFO: Pod "pod-subpath-test-dynamicpv-f7cz": Phase="Pending", Reason="", readiness=false. Elapsed: 2.216027663s
Oct  2 14:56:12.600: INFO: Pod "pod-subpath-test-dynamicpv-f7cz": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.324128971s
STEP: Saw pod success
Oct  2 14:56:12.600: INFO: Pod "pod-subpath-test-dynamicpv-f7cz" satisfied condition "Succeeded or Failed"
Oct  2 14:56:12.707: INFO: Trying to get logs from node k8s-agentpool1-13787793-vmss000001 pod pod-subpath-test-dynamicpv-f7cz container test-container-subpath-dynamicpv-f7cz: <nil>
STEP: delete the pod
Oct  2 14:56:12.931: INFO: Waiting for pod pod-subpath-test-dynamicpv-f7cz to disappear
Oct  2 14:56:13.039: INFO: Pod pod-subpath-test-dynamicpv-f7cz no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-f7cz
Oct  2 14:56:13.039: INFO: Deleting pod "pod-subpath-test-dynamicpv-f7cz" in namespace "provisioning-1687"
... skipping 23 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support existing directories when readOnly specified in the volumeSource
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:395
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directories when readOnly specified in the volumeSource","total":20,"completed":6,"skipped":776,"failed":0}

SSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning 
  should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:201
... skipping 100 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:201
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]","total":20,"completed":7,"skipped":786,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy 
  (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
... skipping 90 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents","total":20,"completed":8,"skipped":812,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath 
  should support readOnly directory specified in the volumeMount
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:365
... skipping 16 lines ...
Oct  2 15:00:56.674: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comhbl7j] to have phase Bound
Oct  2 15:00:56.782: INFO: PersistentVolumeClaim test.csi.azure.comhbl7j found but phase is Pending instead of Bound.
Oct  2 15:00:58.890: INFO: PersistentVolumeClaim test.csi.azure.comhbl7j found but phase is Pending instead of Bound.
Oct  2 15:01:00.998: INFO: PersistentVolumeClaim test.csi.azure.comhbl7j found and phase=Bound (4.323760993s)
STEP: Creating pod pod-subpath-test-dynamicpv-7zs8
STEP: Creating a pod to test subpath
Oct  2 15:01:01.322: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-7zs8" in namespace "provisioning-2365" to be "Succeeded or Failed"
Oct  2 15:01:01.432: INFO: Pod "pod-subpath-test-dynamicpv-7zs8": Phase="Pending", Reason="", readiness=false. Elapsed: 109.07343ms
Oct  2 15:01:03.553: INFO: Pod "pod-subpath-test-dynamicpv-7zs8": Phase="Pending", Reason="", readiness=false. Elapsed: 2.230830333s
Oct  2 15:01:05.662: INFO: Pod "pod-subpath-test-dynamicpv-7zs8": Phase="Pending", Reason="", readiness=false. Elapsed: 4.340011884s
Oct  2 15:01:07.771: INFO: Pod "pod-subpath-test-dynamicpv-7zs8": Phase="Pending", Reason="", readiness=false. Elapsed: 6.448645583s
Oct  2 15:01:09.882: INFO: Pod "pod-subpath-test-dynamicpv-7zs8": Phase="Pending", Reason="", readiness=false. Elapsed: 8.559299107s
Oct  2 15:01:11.991: INFO: Pod "pod-subpath-test-dynamicpv-7zs8": Phase="Pending", Reason="", readiness=false. Elapsed: 10.668460622s
Oct  2 15:01:14.101: INFO: Pod "pod-subpath-test-dynamicpv-7zs8": Phase="Pending", Reason="", readiness=false. Elapsed: 12.778630446s
Oct  2 15:01:16.209: INFO: Pod "pod-subpath-test-dynamicpv-7zs8": Phase="Pending", Reason="", readiness=false. Elapsed: 14.88654365s
Oct  2 15:01:18.317: INFO: Pod "pod-subpath-test-dynamicpv-7zs8": Phase="Pending", Reason="", readiness=false. Elapsed: 16.994750787s
Oct  2 15:01:20.427: INFO: Pod "pod-subpath-test-dynamicpv-7zs8": Phase="Pending", Reason="", readiness=false. Elapsed: 19.10425183s
Oct  2 15:01:22.535: INFO: Pod "pod-subpath-test-dynamicpv-7zs8": Phase="Pending", Reason="", readiness=false. Elapsed: 21.212455855s
Oct  2 15:01:24.643: INFO: Pod "pod-subpath-test-dynamicpv-7zs8": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.3210098s
STEP: Saw pod success
Oct  2 15:01:24.644: INFO: Pod "pod-subpath-test-dynamicpv-7zs8" satisfied condition "Succeeded or Failed"
Oct  2 15:01:24.752: INFO: Trying to get logs from node k8s-agentpool1-13787793-vmss000001 pod pod-subpath-test-dynamicpv-7zs8 container test-container-subpath-dynamicpv-7zs8: <nil>
STEP: delete the pod
Oct  2 15:01:25.028: INFO: Waiting for pod pod-subpath-test-dynamicpv-7zs8 to disappear
Oct  2 15:01:25.135: INFO: Pod pod-subpath-test-dynamicpv-7zs8 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-7zs8
Oct  2 15:01:25.135: INFO: Deleting pod "pod-subpath-test-dynamicpv-7zs8" in namespace "provisioning-2365"
... skipping 23 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support readOnly directory specified in the volumeMount
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:365
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly directory specified in the volumeMount","total":20,"completed":9,"skipped":933,"failed":0}

SSSSSSSSSS
------------------------------
Oct  2 15:02:07.001: INFO: Running AfterSuite actions on all nodes
Oct  2 15:02:07.001: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func17.2
Oct  2 15:02:07.001: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func8.2
... skipping 22 lines ...
Oct  2 14:48:25.013: INFO: Deleting pod "inline-volume-sqz2c" in namespace "ephemeral-6615"
Oct  2 14:48:25.126: INFO: Wait up to 5m0s for pod "inline-volume-sqz2c" to be fully deleted
Oct  2 14:48:25.234: INFO: Creating resource for dynamic PV
Oct  2 14:48:25.234: INFO: Using claimSize:5Gi, test suite supported size:{ }, driver(test.csi.azure.com) supported size:{ } 
STEP: creating a StorageClass ephemeral-6615-e2e-sc6qzx2
STEP: checking the requested inline volume exists in the pod running on node {Name: Selector:map[] Affinity:nil}
Oct  2 15:04:10.230: FAIL: waiting for second pod with inline volume
Unexpected error:
    <*errors.errorString | 0xc00024c250>: {
        s: "timed out waiting for the condition",
    }
    timed out waiting for the condition
occurred

... skipping 46 lines ...
Oct  2 15:05:16.368: INFO: At 2021-10-02 14:49:07 +0000 UTC - event for inline-volume-tester-m64rn: {kubelet k8s-agentpool1-13787793-vmss000000} Started: Started container csi-volume-tester
Oct  2 15:05:16.368: INFO: At 2021-10-02 14:49:07 +0000 UTC - event for inline-volume-tester-m64rn: {kubelet k8s-agentpool1-13787793-vmss000000} Created: Created container csi-volume-tester
Oct  2 15:05:16.368: INFO: At 2021-10-02 14:49:09 +0000 UTC - event for inline-volume-tester2-8ms2w-my-volume-0: {persistentvolume-controller } WaitForPodScheduled: waiting for pod inline-volume-tester2-8ms2w to be scheduled
Oct  2 15:05:16.368: INFO: At 2021-10-02 14:49:11 +0000 UTC - event for inline-volume-tester2-8ms2w-my-volume-0: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "test.csi.azure.com" or manually created by system administrator
Oct  2 15:05:16.368: INFO: At 2021-10-02 14:49:11 +0000 UTC - event for inline-volume-tester2-8ms2w-my-volume-0: {test.csi.azure.com_k8s-agentpool1-13787793-vmss000001_c216fb8a-eb81-41fb-bb7b-2c059211bfa6 } Provisioning: External provisioner is provisioning volume for claim "ephemeral-6615/inline-volume-tester2-8ms2w-my-volume-0"
Oct  2 15:05:16.368: INFO: At 2021-10-02 14:49:13 +0000 UTC - event for inline-volume-tester2-8ms2w-my-volume-0: {test.csi.azure.com_k8s-agentpool1-13787793-vmss000001_c216fb8a-eb81-41fb-bb7b-2c059211bfa6 } ProvisioningSucceeded: Successfully provisioned volume pvc-cc756372-10e6-4c4a-9a17-148d8582402b
Oct  2 15:05:16.368: INFO: At 2021-10-02 14:49:14 +0000 UTC - event for inline-volume-tester2-8ms2w: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-cc756372-10e6-4c4a-9a17-148d8582402b" : rpc error: code = Unknown desc = Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
Oct  2 15:05:16.368: INFO: At 2021-10-02 14:51:17 +0000 UTC - event for inline-volume-tester2-8ms2w: {kubelet k8s-agentpool1-13787793-vmss000000} FailedMount: Unable to attach or mount volumes: unmounted volumes=[my-volume-0], unattached volumes=[my-volume-0 kube-api-access-p8rw9]: timed out waiting for the condition
... skipping 105 lines ...
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support two pods which share the same volume [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:183

    Oct  2 15:04:10.230: waiting for second pod with inline volume
    Unexpected error:
        <*errors.errorString | 0xc00024c250>: {
            s: "timed out waiting for the condition",
        }
        timed out waiting for the condition
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:201
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support two pods which share the same volume","total":16,"completed":1,"skipped":307,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support two pods which share the same volume"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (ext4)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 15:05:23.106: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 142 lines ...
Oct  2 15:05:24.445: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com95bkn] to have phase Bound
Oct  2 15:05:24.554: INFO: PersistentVolumeClaim test.csi.azure.com95bkn found but phase is Pending instead of Bound.
Oct  2 15:05:26.663: INFO: PersistentVolumeClaim test.csi.azure.com95bkn found but phase is Pending instead of Bound.
Oct  2 15:05:28.773: INFO: PersistentVolumeClaim test.csi.azure.com95bkn found and phase=Bound (4.328221786s)
STEP: Creating pod pod-subpath-test-dynamicpv-64xw
STEP: Creating a pod to test subpath
Oct  2 15:05:29.111: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-64xw" in namespace "provisioning-1422" to be "Succeeded or Failed"
Oct  2 15:05:29.220: INFO: Pod "pod-subpath-test-dynamicpv-64xw": Phase="Pending", Reason="", readiness=false. Elapsed: 108.610612ms
Oct  2 15:05:31.329: INFO: Pod "pod-subpath-test-dynamicpv-64xw": Phase="Pending", Reason="", readiness=false. Elapsed: 2.2184377s
Oct  2 15:05:33.440: INFO: Pod "pod-subpath-test-dynamicpv-64xw": Phase="Pending", Reason="", readiness=false. Elapsed: 4.328647056s
Oct  2 15:05:35.549: INFO: Pod "pod-subpath-test-dynamicpv-64xw": Phase="Pending", Reason="", readiness=false. Elapsed: 6.438129605s
Oct  2 15:05:37.659: INFO: Pod "pod-subpath-test-dynamicpv-64xw": Phase="Pending", Reason="", readiness=false. Elapsed: 8.547679171s
Oct  2 15:05:39.768: INFO: Pod "pod-subpath-test-dynamicpv-64xw": Phase="Pending", Reason="", readiness=false. Elapsed: 10.65704181s
Oct  2 15:05:41.878: INFO: Pod "pod-subpath-test-dynamicpv-64xw": Phase="Pending", Reason="", readiness=false. Elapsed: 12.767063079s
Oct  2 15:05:43.987: INFO: Pod "pod-subpath-test-dynamicpv-64xw": Phase="Pending", Reason="", readiness=false. Elapsed: 14.87598516s
Oct  2 15:05:46.098: INFO: Pod "pod-subpath-test-dynamicpv-64xw": Phase="Pending", Reason="", readiness=false. Elapsed: 16.987334078s
Oct  2 15:05:48.214: INFO: Pod "pod-subpath-test-dynamicpv-64xw": Phase="Pending", Reason="", readiness=false. Elapsed: 19.103358903s
Oct  2 15:05:50.333: INFO: Pod "pod-subpath-test-dynamicpv-64xw": Phase="Pending", Reason="", readiness=false. Elapsed: 21.222025099s
Oct  2 15:05:52.442: INFO: Pod "pod-subpath-test-dynamicpv-64xw": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.331374032s
STEP: Saw pod success
Oct  2 15:05:52.442: INFO: Pod "pod-subpath-test-dynamicpv-64xw" satisfied condition "Succeeded or Failed"
Oct  2 15:05:52.551: INFO: Trying to get logs from node k8s-agentpool1-13787793-vmss000001 pod pod-subpath-test-dynamicpv-64xw container test-container-subpath-dynamicpv-64xw: <nil>
STEP: delete the pod
Oct  2 15:05:52.800: INFO: Waiting for pod pod-subpath-test-dynamicpv-64xw to disappear
Oct  2 15:05:52.908: INFO: Pod pod-subpath-test-dynamicpv-64xw no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-64xw
Oct  2 15:05:52.908: INFO: Deleting pod "pod-subpath-test-dynamicpv-64xw" in namespace "provisioning-1422"
... skipping 23 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support readOnly file specified in the volumeMount [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:380
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly file specified in the volumeMount [LinuxOnly]","total":16,"completed":2,"skipped":813,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support two pods which share the same volume"]}

SSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: CSI Ephemeral-volume (default fs)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 15:06:34.813: INFO: Driver "test.csi.azure.com" does not support volume type "CSIInlineVolume" - skipping
... skipping 148 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support volume limits [Serial]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumelimits.go:123
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits should support volume limits [Serial]","total":15,"completed":2,"skipped":753,"failed":0}

SSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral 
  should create read-only inline ephemeral volume
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149
... skipping 43 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read-only inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read-only inline ephemeral volume","total":15,"completed":3,"skipped":766,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (ntfs)][Feature:Windows] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  2 15:14:46.652: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 36 lines ...
Oct  2 15:14:46.709: INFO: Running AfterSuite actions on node 1



Summarizing 1 Failure:

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral [Measurement] should support two pods which share the same volume 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:201

Ran 30 of 6664 Specs in 1712.501 seconds
FAIL! -- 29 Passed | 1 Failed | 0 Pending | 6634 Skipped 

Ginkgo ran 1 suite in 28m35.414057904s
Test Suite Failed
+ print_logs
+ sed -i s/disk.csi.azure.com/test.csi.azure.com/g deploy/example/storageclass-azuredisk-csi.yaml
+ bash ./hack/verify-examples.sh linux azurepubliccloud ephemeral test
begin to create deployment examples ...
storageclass.storage.k8s.io/managed-csi created
Applying config "deploy/example/deployment.yaml"
... skipping 73 lines ...
Platform: linux/amd64
Topology Key: topology.test.csi.azure.com/zone

Streaming logs below:
I1002 14:45:47.087394       1 azuredisk.go:139] driver userAgent: test.csi.azure.com/v1.8.0-b1f99a5a1f102374a7a848cd09676c7bf2753657 gc/go1.17.1 (amd64-linux) e2e-test
I1002 14:45:47.088925       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W1002 14:45:47.190740       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I1002 14:45:47.190807       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I1002 14:45:47.190833       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I1002 14:45:47.190878       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I1002 14:45:47.191616       1 azure_auth.go:234] Using AzurePublicCloud environment
I1002 14:45:47.191639       1 azure_auth.go:96] azure: using managed identity extension to retrieve access token
I1002 14:45:47.191644       1 azure_auth.go:102] azure: using User Assigned MSI ID to retrieve access token
... skipping 91 lines ...
I1002 14:46:17.256225       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:46:17.256249       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-dcfd2693-1b8f-411e-b263-be00dadb544b","parameters":{"csi.storage.k8s.io/pv/name":"pvc-dcfd2693-1b8f-411e-b263-be00dadb544b","csi.storage.k8s.io/pvc/name":"test.csi.azure.comlh755","csi.storage.k8s.io/pvc/namespace":"volumemode-7949"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:46:17.380825       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:46:17.380860       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-9c7b4d90-90c3-413c-b8de-6c97ad300b76","parameters":{"csi.storage.k8s.io/pv/name":"pvc-9c7b4d90-90c3-413c-b8de-6c97ad300b76","csi.storage.k8s.io/pvc/name":"test.csi.azure.com4vvb9","csi.storage.k8s.io/pvc/namespace":"provisioning-4498"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:46:17.481764       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:46:17.481795       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-cac27315-0864-48af-8150-04b3cf4fdfb3","parameters":{"csi.storage.k8s.io/pv/name":"pvc-cac27315-0864-48af-8150-04b3cf4fdfb3","csi.storage.k8s.io/pvc/name":"test.csi.azure.com6x7zh","csi.storage.k8s.io/pvc/namespace":"volume-5831"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:46:17.580720       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dcfd2693-1b8f-411e-b263-be00dadb544b, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-dcfd2693-1b8f-411e-b263-be00dadb544b' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:46:17.580940       1 controllerserver.go:271] begin to create azure disk(pvc-dcfd2693-1b8f-411e-b263-be00dadb544b) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:46:17.580962       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-dcfd2693-1b8f-411e-b263-be00dadb544b StorageAccountType:StandardSSD_LRS Size:5
I1002 14:46:17.680753       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a414f456-3b4d-492f-9c71-72963857ef6a, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-a414f456-3b4d-492f-9c71-72963857ef6a' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:46:17.680776       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9c7b4d90-90c3-413c-b8de-6c97ad300b76, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-9c7b4d90-90c3-413c-b8de-6c97ad300b76' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:46:17.680844       1 controllerserver.go:271] begin to create azure disk(pvc-a414f456-3b4d-492f-9c71-72963857ef6a) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:46:17.680853       1 controllerserver.go:271] begin to create azure disk(pvc-9c7b4d90-90c3-413c-b8de-6c97ad300b76) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:46:17.680872       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-a414f456-3b4d-492f-9c71-72963857ef6a StorageAccountType:StandardSSD_LRS Size:5
I1002 14:46:17.680877       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-9c7b4d90-90c3-413c-b8de-6c97ad300b76 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:46:17.706848       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cac27315-0864-48af-8150-04b3cf4fdfb3, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-cac27315-0864-48af-8150-04b3cf4fdfb3' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:46:17.706937       1 controllerserver.go:271] begin to create azure disk(pvc-cac27315-0864-48af-8150-04b3cf4fdfb3) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:46:17.707631       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-cac27315-0864-48af-8150-04b3cf4fdfb3 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:46:18.180656       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:46:18.180684       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-03f3f456-8874-43a6-8c39-e4777459d5ae","parameters":{"csi.storage.k8s.io/pv/name":"pvc-03f3f456-8874-43a6-8c39-e4777459d5ae","csi.storage.k8s.io/pvc/name":"inline-volume-tester-28jtf-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2593"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:46:18.226321       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-03f3f456-8874-43a6-8c39-e4777459d5ae, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-03f3f456-8874-43a6-8c39-e4777459d5ae' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:46:18.226421       1 controllerserver.go:271] begin to create azure disk(pvc-03f3f456-8874-43a6-8c39-e4777459d5ae) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:46:18.226438       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-03f3f456-8874-43a6-8c39-e4777459d5ae StorageAccountType:StandardSSD_LRS Size:5
I1002 14:46:18.592896       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:46:18.592928       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-fc6193cd-513b-450d-851f-4943d7672a69","parameters":{"csi.storage.k8s.io/pv/name":"pvc-fc6193cd-513b-450d-851f-4943d7672a69","csi.storage.k8s.io/pvc/name":"test.csi.azure.comghtv9","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-5574"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:46:18.609542       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-fc6193cd-513b-450d-851f-4943d7672a69, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-fc6193cd-513b-450d-851f-4943d7672a69' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:46:18.609660       1 controllerserver.go:271] begin to create azure disk(pvc-fc6193cd-513b-450d-851f-4943d7672a69) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:46:18.609695       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-fc6193cd-513b-450d-851f-4943d7672a69 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:46:19.566799       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:46:19.566826       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-ff455d53-5b23-4340-9445-1e827d77728b","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ff455d53-5b23-4340-9445-1e827d77728b","csi.storage.k8s.io/pvc/name":"inline-volume-tester-zxkjp-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-9702"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:46:19.611946       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ff455d53-5b23-4340-9445-1e827d77728b, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ff455d53-5b23-4340-9445-1e827d77728b' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:46:19.612068       1 controllerserver.go:271] begin to create azure disk(pvc-ff455d53-5b23-4340-9445-1e827d77728b) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:46:19.612109       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ff455d53-5b23-4340-9445-1e827d77728b StorageAccountType:StandardSSD_LRS Size:5
I1002 14:46:20.026414       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-dcfd2693-1b8f-411e-b263-be00dadb544b StorageAccountType:StandardSSD_LRS Size:5
I1002 14:46:20.026486       1 controllerserver.go:348] create azure disk(pvc-dcfd2693-1b8f-411e-b263-be00dadb544b) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-dcfd2693-1b8f-411e-b263-be00dadb544b kubernetes.io-created-for-pvc-name:test.csi.azure.comlh755 kubernetes.io-created-for-pvc-namespace:volumemode-7949]) successfully
I1002 14:46:20.026517       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-dcfd2693-1b8f-411e-b263-be00dadb544b","csi.storage.k8s.io/pvc/name":"test.csi.azure.comlh755","csi.storage.k8s.io/pvc/namespace":"volumemode-7949","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dcfd2693-1b8f-411e-b263-be00dadb544b"}}
I1002 14:46:20.052186       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-9c7b4d90-90c3-413c-b8de-6c97ad300b76 StorageAccountType:StandardSSD_LRS Size:5
... skipping 49 lines ...
I1002 14:46:22.330633       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9c7b4d90-90c3-413c-b8de-6c97ad300b76:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9c7b4d90-90c3-413c-b8de-6c97ad300b76  true false 0})])
I1002 14:46:23.061838       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:46:23.061869       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ff455d53-5b23-4340-9445-1e827d77728b","csi.storage.k8s.io/pvc/name":"inline-volume-tester-zxkjp-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-9702","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ff455d53-5b23-4340-9445-1e827d77728b"}
I1002 14:46:23.085068       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ff455d53-5b23-4340-9445-1e827d77728b" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:46:23.085107       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:46:23.085131       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ff455d53-5b23-4340-9445-1e827d77728b" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:46:23.094509       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cac27315-0864-48af-8150-04b3cf4fdfb3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cac27315-0864-48af-8150-04b3cf4fdfb3  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:46:23.094720       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:46:23.094772       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:46:23.134339       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9c7b4d90-90c3-413c-b8de-6c97ad300b76:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9c7b4d90-90c3-413c-b8de-6c97ad300b76  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:46:23.139510       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a414f456-3b4d-492f-9c71-72963857ef6a" lun 1 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-03f3f456-8874-43a6-8c39-e4777459d5ae:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-03f3f456-8874-43a6-8c39-e4777459d5ae  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a414f456-3b4d-492f-9c71-72963857ef6a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a414f456-3b4d-492f-9c71-72963857ef6a  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ff455d53-5b23-4340-9445-1e827d77728b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ff455d53-5b23-4340-9445-1e827d77728b  true false 3})]
I1002 14:46:23.139542       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:46:23.139578       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-03f3f456-8874-43a6-8c39-e4777459d5ae:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-03f3f456-8874-43a6-8c39-e4777459d5ae  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a414f456-3b4d-492f-9c71-72963857ef6a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a414f456-3b4d-492f-9c71-72963857ef6a  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ff455d53-5b23-4340-9445-1e827d77728b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ff455d53-5b23-4340-9445-1e827d77728b  true false 3})])
I1002 14:46:23.139671       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:46:23.139706       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:46:23.192700       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dcfd2693-1b8f-411e-b263-be00dadb544b" lun 1 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dcfd2693-1b8f-411e-b263-be00dadb544b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dcfd2693-1b8f-411e-b263-be00dadb544b  true false 1})]
I1002 14:46:23.192745       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:46:23.192767       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dcfd2693-1b8f-411e-b263-be00dadb544b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dcfd2693-1b8f-411e-b263-be00dadb544b  true false 1})])
I1002 14:46:23.264917       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:46:23.264943       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-fc6193cd-513b-450d-851f-4943d7672a69","csi.storage.k8s.io/pvc/name":"test.csi.azure.comghtv9","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-5574","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-fc6193cd-513b-450d-851f-4943d7672a69"}
I1002 14:46:23.285104       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-fc6193cd-513b-450d-851f-4943d7672a69" to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:46:23.285136       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:46:23.285159       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-fc6193cd-513b-450d-851f-4943d7672a69" to node "k8s-agentpool1-13787793-vmss000001"
I1002 14:46:23.338285       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-03f3f456-8874-43a6-8c39-e4777459d5ae:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-03f3f456-8874-43a6-8c39-e4777459d5ae  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a414f456-3b4d-492f-9c71-72963857ef6a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a414f456-3b4d-492f-9c71-72963857ef6a  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ff455d53-5b23-4340-9445-1e827d77728b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ff455d53-5b23-4340-9445-1e827d77728b  true false 3})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:46:23.338425       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:46:23.338498       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:46:23.357547       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dcfd2693-1b8f-411e-b263-be00dadb544b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dcfd2693-1b8f-411e-b263-be00dadb544b  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:46:23.380574       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-03f3f456-8874-43a6-8c39-e4777459d5ae" lun 2 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[]
I1002 14:46:23.380625       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-03f3f456-8874-43a6-8c39-e4777459d5ae" attached to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:46:23.380653       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-03f3f456-8874-43a6-8c39-e4777459d5ae" to node "k8s-agentpool1-13787793-vmss000000" successfully
I1002 14:46:23.380676       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"2"}}
I1002 14:46:23.380816       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:46:23.380839       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 9 lines ...
I1002 14:46:23.423982       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-fc6193cd-513b-450d-851f-4943d7672a69" lun 2 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-fc6193cd-513b-450d-851f-4943d7672a69:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc6193cd-513b-450d-851f-4943d7672a69  true false 2})]
I1002 14:46:23.424033       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:46:23.424074       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-fc6193cd-513b-450d-851f-4943d7672a69:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc6193cd-513b-450d-851f-4943d7672a69  true false 2})])
I1002 14:46:23.424195       1 azure_controller_common.go:457] azureDisk - find disk: lun 3 name "pvc-ff455d53-5b23-4340-9445-1e827d77728b" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ff455d53-5b23-4340-9445-1e827d77728b"
I1002 14:46:23.424241       1 controllerserver.go:455] Attach operation is successful. volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ff455d53-5b23-4340-9445-1e827d77728b" is already attached to node "k8s-agentpool1-13787793-vmss000000" at lun 3.
I1002 14:46:23.424275       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"3"}}
I1002 14:46:23.616381       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-fc6193cd-513b-450d-851f-4943d7672a69:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc6193cd-513b-450d-851f-4943d7672a69  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:46:36.592329       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:46:36.592357       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462","parameters":{"csi.storage.k8s.io/pv/name":"pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-bnsnw-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2593"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:46:36.608995       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:46:36.609077       1 controllerserver.go:271] begin to create azure disk(pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:46:36.609092       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:46:38.189863       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cac27315-0864-48af-8150-04b3cf4fdfb3" attached to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:46:38.189903       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cac27315-0864-48af-8150-04b3cf4fdfb3" to node "k8s-agentpool1-13787793-vmss000000" successfully
I1002 14:46:38.189924       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1002 14:46:38.195758       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 40 lines ...
I1002 14:46:40.086575       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:46:40.086602       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:46:40.086618       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:46:40.086631       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462" lun 4 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462  true false 4})]
I1002 14:46:40.086655       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:46:40.086671       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462  true false 4})])
I1002 14:46:40.229686       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462  true false 4})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:46:47.825550       1 azure_vmss.go:319] Couldn't find VMSS VM with scaleSetName "k8s-agentpool1-13787793-vmss" and instanceID "0", refreshing the cache
I1002 14:46:55.443345       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462" attached to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:46:55.443430       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462" to node "k8s-agentpool1-13787793-vmss000000" successfully
I1002 14:46:55.443455       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"4"}}
I1002 14:46:55.451873       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:46:55.451893       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-bnsnw-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2593","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462"}
... skipping 38 lines ...
I1002 14:47:18.515646       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-fc6193cd-513b-450d-851f-4943d7672a69"}
I1002 14:47:18.515725       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-fc6193cd-513b-450d-851f-4943d7672a69 from node k8s-agentpool1-13787793-vmss000001
I1002 14:47:18.515739       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:47:18.515777       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:47:22.281485       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:47:22.281512       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-d30a4bd0-a617-497e-836d-c1820e36efa5","parameters":{"csi.storage.k8s.io/pv/name":"pvc-d30a4bd0-a617-497e-836d-c1820e36efa5","csi.storage.k8s.io/pvc/name":"test.csi.azure.comlvxcq","csi.storage.k8s.io/pvc/namespace":"volumemode-2748"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":7}}]}
I1002 14:47:22.300955       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d30a4bd0-a617-497e-836d-c1820e36efa5, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-d30a4bd0-a617-497e-836d-c1820e36efa5' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:47:22.301020       1 controllerserver.go:271] begin to create azure disk(pvc-d30a4bd0-a617-497e-836d-c1820e36efa5) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:47:22.301036       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-d30a4bd0-a617-497e-836d-c1820e36efa5 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:47:24.664710       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-d30a4bd0-a617-497e-836d-c1820e36efa5 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:47:24.664797       1 controllerserver.go:348] create azure disk(pvc-d30a4bd0-a617-497e-836d-c1820e36efa5) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-d30a4bd0-a617-497e-836d-c1820e36efa5 kubernetes.io-created-for-pvc-name:test.csi.azure.comlvxcq kubernetes.io-created-for-pvc-namespace:volumemode-2748]) successfully
I1002 14:47:24.664820       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d30a4bd0-a617-497e-836d-c1820e36efa5","csi.storage.k8s.io/pvc/name":"test.csi.azure.comlvxcq","csi.storage.k8s.io/pvc/namespace":"volumemode-2748","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d30a4bd0-a617-497e-836d-c1820e36efa5"}}
I1002 14:47:26.993002       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 41 lines ...
I1002 14:47:41.323849       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cac27315-0864-48af-8150-04b3cf4fdfb3) returned with <nil>
I1002 14:47:41.323866       1 utils.go:102] GRPC response: {}
I1002 14:47:47.054619       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:47:47.054671       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b","parameters":{"csi.storage.k8s.io/pv/name":"pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b","csi.storage.k8s.io/pvc/name":"inline-volume-tester-scdvp-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-6328"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:47:47.065145       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:47:47.065170       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4","parameters":{"csi.storage.k8s.io/pv/name":"pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4","csi.storage.k8s.io/pvc/name":"inline-volume-tester-scdvp-my-volume-1","csi.storage.k8s.io/pvc/namespace":"ephemeral-6328"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:47:47.073766       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:47:47.073864       1 controllerserver.go:271] begin to create azure disk(pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:47:47.073880       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b StorageAccountType:StandardSSD_LRS Size:5
I1002 14:47:47.082550       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:47:47.082614       1 controllerserver.go:271] begin to create azure disk(pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:47:47.082625       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:47:49.393667       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b StorageAccountType:StandardSSD_LRS Size:5
I1002 14:47:49.393712       1 controllerserver.go:348] create azure disk(pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b kubernetes.io-created-for-pvc-name:inline-volume-tester-scdvp-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-6328]) successfully
I1002 14:47:49.393744       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b","csi.storage.k8s.io/pvc/name":"inline-volume-tester-scdvp-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-6328","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b"}}
I1002 14:47:49.408052       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:47:49.408156       1 controllerserver.go:348] create azure disk(pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4 kubernetes.io-created-for-pvc-name:inline-volume-tester-scdvp-my-volume-1 kubernetes.io-created-for-pvc-namespace:ephemeral-6328]) successfully
I1002 14:47:49.408193       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4","csi.storage.k8s.io/pvc/name":"inline-volume-tester-scdvp-my-volume-1","csi.storage.k8s.io/pvc/namespace":"ephemeral-6328","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4"}}
I1002 14:47:50.056713       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:47:50.056739       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855","parameters":{"csi.storage.k8s.io/pv/name":"pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855","csi.storage.k8s.io/pvc/name":"inline-volume-tester-tmvzc-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2325"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:47:50.073965       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:47:50.074142       1 controllerserver.go:271] begin to create azure disk(pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:47:50.074202       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:47:50.089790       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:47:50.089867       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4","csi.storage.k8s.io/pvc/name":"inline-volume-tester-scdvp-my-volume-1","csi.storage.k8s.io/pvc/namespace":"ephemeral-6328","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4"}
I1002 14:47:50.106237       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:47:50.106258       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b","csi.storage.k8s.io/pvc/name":"inline-volume-tester-scdvp-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-6328","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b"}
... skipping 25 lines ...
I1002 14:47:54.228535       1 controllerserver.go:533] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ff455d53-5b23-4340-9445-1e827d77728b from node k8s-agentpool1-13787793-vmss000000 successfully
I1002 14:47:54.228547       1 utils.go:102] GRPC response: {}
I1002 14:47:54.228643       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a414f456-3b4d-492f-9c71-72963857ef6a" from node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cac27315-0864-48af-8150-04b3cf4fdfb3:pvc-cac27315-0864-48af-8150-04b3cf4fdfb3]
I1002 14:47:54.228690       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
E1002 14:47:54.271810       1 azure_controller_vmss.go:183] detach azure disk on node(k8s-agentpool1-13787793-vmss000000): disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cac27315-0864-48af-8150-04b3cf4fdfb3:pvc-cac27315-0864-48af-8150-04b3cf4fdfb3]) not found
I1002 14:47:54.271842       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cac27315-0864-48af-8150-04b3cf4fdfb3:pvc-cac27315-0864-48af-8150-04b3cf4fdfb3])
I1002 14:47:54.397068       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855  true false 3}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-d30a4bd0-a617-497e-836d-c1820e36efa5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d30a4bd0-a617-497e-836d-c1820e36efa5  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:47:54.397421       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:47:54.397452       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:47:54.443778       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4" lun 1 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[]
I1002 14:47:54.443864       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:47:54.443878       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 14:47:54.443893       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
... skipping 56 lines ...
I1002 14:48:18.294056       1 utils.go:102] GRPC response: {}
I1002 14:48:18.530624       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ff455d53-5b23-4340-9445-1e827d77728b
I1002 14:48:18.530668       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ff455d53-5b23-4340-9445-1e827d77728b) returned with <nil>
I1002 14:48:18.530710       1 utils.go:102] GRPC response: {}
I1002 14:48:24.523042       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:24.523068       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-7112c99e-555c-4139-ba7f-ae80c671397d","parameters":{"csi.storage.k8s.io/pv/name":"pvc-7112c99e-555c-4139-ba7f-ae80c671397d","csi.storage.k8s.io/pvc/name":"test.csi.azure.compc4nk","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:24.539914       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7112c99e-555c-4139-ba7f-ae80c671397d, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-7112c99e-555c-4139-ba7f-ae80c671397d' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:24.539988       1 controllerserver.go:271] begin to create azure disk(pvc-7112c99e-555c-4139-ba7f-ae80c671397d) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:24.540022       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-7112c99e-555c-4139-ba7f-ae80c671397d StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:26.874335       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-7112c99e-555c-4139-ba7f-ae80c671397d StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:26.874382       1 controllerserver.go:348] create azure disk(pvc-7112c99e-555c-4139-ba7f-ae80c671397d) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-7112c99e-555c-4139-ba7f-ae80c671397d kubernetes.io-created-for-pvc-name:test.csi.azure.compc4nk kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:26.874408       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7112c99e-555c-4139-ba7f-ae80c671397d","csi.storage.k8s.io/pvc/name":"test.csi.azure.compc4nk","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7112c99e-555c-4139-ba7f-ae80c671397d"}}
I1002 14:48:26.977090       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1002 14:48:26.977116       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a414f456-3b4d-492f-9c71-72963857ef6a"}
I1002 14:48:26.977197       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a414f456-3b4d-492f-9c71-72963857ef6a)
I1002 14:48:27.081507       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:27.081535       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3","parameters":{"csi.storage.k8s.io/pv/name":"pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3","csi.storage.k8s.io/pvc/name":"inline-volume-tester-m64rn-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-6615"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:27.104206       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:27.104342       1 controllerserver.go:271] begin to create azure disk(pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:27.104364       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:29.171770       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:29.171813       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21","csi.storage.k8s.io/pvc/name":"pvc-lff45","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:29.193021       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:29.193088       1 controllerserver.go:271] begin to create azure disk(pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:29.193104       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:29.284622       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:29.284649       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0","parameters":{"csi.storage.k8s.io/pv/name":"pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0","csi.storage.k8s.io/pvc/name":"pvc-82jmj","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:29.302577       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:29.302647       1 controllerserver.go:271] begin to create azure disk(pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:29.302663       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:29.396495       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:29.396536       1 controllerserver.go:348] create azure disk(pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3 kubernetes.io-created-for-pvc-name:inline-volume-tester-m64rn-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-6615]) successfully
I1002 14:48:29.396560       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3","csi.storage.k8s.io/pvc/name":"inline-volume-tester-m64rn-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-6615","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3"}}
I1002 14:48:29.400784       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:29.400809       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-dd56b38f-7605-45b2-879d-e35a4eec8723","parameters":{"csi.storage.k8s.io/pv/name":"pvc-dd56b38f-7605-45b2-879d-e35a4eec8723","csi.storage.k8s.io/pvc/name":"pvc-5c44d","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:29.418716       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:29.418791       1 controllerserver.go:271] begin to create azure disk(pvc-dd56b38f-7605-45b2-879d-e35a4eec8723) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:29.418819       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-dd56b38f-7605-45b2-879d-e35a4eec8723 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:29.514953       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:29.514981       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f","parameters":{"csi.storage.k8s.io/pv/name":"pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f","csi.storage.k8s.io/pvc/name":"pvc-b8xqq","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:29.533044       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:29.533123       1 controllerserver.go:271] begin to create azure disk(pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:29.533140       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:29.615176       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:29.615202       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49","csi.storage.k8s.io/pvc/name":"pvc-r8rlz","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:29.698042       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:29.698122       1 controllerserver.go:271] begin to create azure disk(pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:29.698154       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:29.722112       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:29.722137       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-8f21becf-3e62-48ca-977c-2514cf9f135f","parameters":{"csi.storage.k8s.io/pv/name":"pvc-8f21becf-3e62-48ca-977c-2514cf9f135f","csi.storage.k8s.io/pvc/name":"pvc-c586c","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:29.766178       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8f21becf-3e62-48ca-977c-2514cf9f135f, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-8f21becf-3e62-48ca-977c-2514cf9f135f' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:29.766255       1 controllerserver.go:271] begin to create azure disk(pvc-8f21becf-3e62-48ca-977c-2514cf9f135f) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:29.766272       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-8f21becf-3e62-48ca-977c-2514cf9f135f StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:29.833177       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:29.833211       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-9015e829-c629-45b5-89bc-e6defff725a9","parameters":{"csi.storage.k8s.io/pv/name":"pvc-9015e829-c629-45b5-89bc-e6defff725a9","csi.storage.k8s.io/pvc/name":"pvc-cpzk9","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:29.851533       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:29.851593       1 controllerserver.go:271] begin to create azure disk(pvc-9015e829-c629-45b5-89bc-e6defff725a9) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:29.851606       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-9015e829-c629-45b5-89bc-e6defff725a9 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:29.939304       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:29.939328       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f","parameters":{"csi.storage.k8s.io/pv/name":"pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f","csi.storage.k8s.io/pvc/name":"pvc-8j8n7","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:29.961559       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:29.961654       1 controllerserver.go:271] begin to create azure disk(pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:29.961668       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:30.047039       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:30.047081       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d","parameters":{"csi.storage.k8s.io/pv/name":"pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d","csi.storage.k8s.io/pvc/name":"pvc-czkqn","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:30.065707       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:30.065805       1 controllerserver.go:271] begin to create azure disk(pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:30.065823       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:30.154998       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:30.155023       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17","parameters":{"csi.storage.k8s.io/pv/name":"pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17","csi.storage.k8s.io/pvc/name":"pvc-5qsvc","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:30.159293       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:48:30.159315       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3","csi.storage.k8s.io/pvc/name":"inline-volume-tester-m64rn-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-6615","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3"}
I1002 14:48:30.174539       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:30.174639       1 controllerserver.go:271] begin to create azure disk(pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:30.174653       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:30.179433       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:48:30.179479       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:30.179501       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:48:30.264638       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:30.264665       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9","parameters":{"csi.storage.k8s.io/pv/name":"pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9","csi.storage.k8s.io/pvc/name":"pvc-8qnmx","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:30.282268       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:30.282335       1 controllerserver.go:271] begin to create azure disk(pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:30.282350       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:30.373211       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:30.373235       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-2ada027c-fb64-42ef-9797-f62647ebd42e","parameters":{"csi.storage.k8s.io/pv/name":"pvc-2ada027c-fb64-42ef-9797-f62647ebd42e","csi.storage.k8s.io/pvc/name":"pvc-mzlpl","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:30.397276       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:30.397341       1 controllerserver.go:271] begin to create azure disk(pvc-2ada027c-fb64-42ef-9797-f62647ebd42e) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:30.397361       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-2ada027c-fb64-42ef-9797-f62647ebd42e StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:30.481845       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:30.481872       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b","parameters":{"csi.storage.k8s.io/pv/name":"pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b","csi.storage.k8s.io/pvc/name":"pvc-q5gzm","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:30.503274       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:30.503368       1 controllerserver.go:271] begin to create azure disk(pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:30.503383       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:30.588855       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:30.588889       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606","parameters":{"csi.storage.k8s.io/pv/name":"pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606","csi.storage.k8s.io/pvc/name":"pvc-lwrht","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:30.606628       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:30.606710       1 controllerserver.go:271] begin to create azure disk(pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:30.606732       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:30.696967       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:30.696994       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-e19708a1-7049-4d60-a087-03a7611a7f3b","parameters":{"csi.storage.k8s.io/pv/name":"pvc-e19708a1-7049-4d60-a087-03a7611a7f3b","csi.storage.k8s.io/pvc/name":"pvc-mwx4q","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:30.714933       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:30.715046       1 controllerserver.go:271] begin to create azure disk(pvc-e19708a1-7049-4d60-a087-03a7611a7f3b) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:30.715066       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-e19708a1-7049-4d60-a087-03a7611a7f3b StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:30.809301       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:30.809328       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-9fa24158-b01a-4725-841c-e55266a60e43","parameters":{"csi.storage.k8s.io/pv/name":"pvc-9fa24158-b01a-4725-841c-e55266a60e43","csi.storage.k8s.io/pvc/name":"pvc-m2qqs","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:30.825891       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:30.825991       1 controllerserver.go:271] begin to create azure disk(pvc-9fa24158-b01a-4725-841c-e55266a60e43) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:30.826090       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-9fa24158-b01a-4725-841c-e55266a60e43 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:30.917129       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:30.917153       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0","csi.storage.k8s.io/pvc/name":"pvc-5scjq","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:30.934822       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:30.934912       1 controllerserver.go:271] begin to create azure disk(pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:30.934933       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:31.026657       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:31.026682       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-ad89e320-0715-45d1-a22c-177c4c9109c3","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ad89e320-0715-45d1-a22c-177c4c9109c3","csi.storage.k8s.io/pvc/name":"pvc-sbnzl","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:31.045378       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:31.045493       1 controllerserver.go:271] begin to create azure disk(pvc-ad89e320-0715-45d1-a22c-177c4c9109c3) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:31.045507       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ad89e320-0715-45d1-a22c-177c4c9109c3 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:31.135206       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:31.135231       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0","parameters":{"csi.storage.k8s.io/pv/name":"pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0","csi.storage.k8s.io/pvc/name":"pvc-jdtg5","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:31.153480       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:31.153589       1 controllerserver.go:271] begin to create azure disk(pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:31.153605       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:31.245058       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:31.245082       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-224ff370-4c43-4738-974f-c260f23439dc","parameters":{"csi.storage.k8s.io/pv/name":"pvc-224ff370-4c43-4738-974f-c260f23439dc","csi.storage.k8s.io/pvc/name":"pvc-5cp44","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:31.262746       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:31.262815       1 controllerserver.go:271] begin to create azure disk(pvc-224ff370-4c43-4738-974f-c260f23439dc) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:31.262840       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-224ff370-4c43-4738-974f-c260f23439dc StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:31.353447       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:31.353471       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40","parameters":{"csi.storage.k8s.io/pv/name":"pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40","csi.storage.k8s.io/pvc/name":"pvc-6n2pm","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:31.372152       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:31.372275       1 controllerserver.go:271] begin to create azure disk(pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:31.372295       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:31.465844       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:31.465867       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742","parameters":{"csi.storage.k8s.io/pv/name":"pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742","csi.storage.k8s.io/pvc/name":"pvc-qs8k9","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:31.487579       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:31.487644       1 controllerserver.go:271] begin to create azure disk(pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:31.487662       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:31.551024       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:31.551062       1 controllerserver.go:348] create azure disk(pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21 kubernetes.io-created-for-pvc-name:pvc-lff45 kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:31.551086       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21","csi.storage.k8s.io/pvc/name":"pvc-lff45","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21"}}
I1002 14:48:31.580267       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:31.580299       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-b9075cab-972d-44e5-b5f5-817883de636f","parameters":{"csi.storage.k8s.io/pv/name":"pvc-b9075cab-972d-44e5-b5f5-817883de636f","csi.storage.k8s.io/pvc/name":"pvc-mp5xf","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:31.581347       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-d30a4bd0-a617-497e-836d-c1820e36efa5:pvc-d30a4bd0-a617-497e-836d-c1820e36efa5]) returned with <nil>
I1002 14:48:31.581404       1 azure_controller_common.go:369] azureDisk - detach disk(pvc-d30a4bd0-a617-497e-836d-c1820e36efa5, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d30a4bd0-a617-497e-836d-c1820e36efa5) succeeded
I1002 14:48:31.581416       1 controllerserver.go:533] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d30a4bd0-a617-497e-836d-c1820e36efa5 from node k8s-agentpool1-13787793-vmss000001 successfully
I1002 14:48:31.581426       1 utils.go:102] GRPC response: {}
I1002 14:48:31.609023       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-b9075cab-972d-44e5-b5f5-817883de636f, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-b9075cab-972d-44e5-b5f5-817883de636f' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:31.609122       1 controllerserver.go:271] begin to create azure disk(pvc-b9075cab-972d-44e5-b5f5-817883de636f) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:31.609141       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-b9075cab-972d-44e5-b5f5-817883de636f StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:31.689184       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:31.689210       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702","parameters":{"csi.storage.k8s.io/pv/name":"pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702","csi.storage.k8s.io/pvc/name":"pvc-jb4gl","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:31.708079       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:31.708206       1 controllerserver.go:271] begin to create azure disk(pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:31.708220       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:31.743563       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-dd56b38f-7605-45b2-879d-e35a4eec8723 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:31.743622       1 controllerserver.go:348] create azure disk(pvc-dd56b38f-7605-45b2-879d-e35a4eec8723) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-dd56b38f-7605-45b2-879d-e35a4eec8723 kubernetes.io-created-for-pvc-name:pvc-5c44d kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:31.743650       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-dd56b38f-7605-45b2-879d-e35a4eec8723","csi.storage.k8s.io/pvc/name":"pvc-5c44d","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723"}}
I1002 14:48:31.880805       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:31.880892       1 controllerserver.go:348] create azure disk(pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0 kubernetes.io-created-for-pvc-name:pvc-82jmj kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:31.880938       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0","csi.storage.k8s.io/pvc/name":"pvc-82jmj","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0"}}
I1002 14:48:31.899425       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:31.899451       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d","parameters":{"csi.storage.k8s.io/pv/name":"pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d","csi.storage.k8s.io/pvc/name":"pvc-s8t9r","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:31.916914       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:31.917007       1 controllerserver.go:271] begin to create azure disk(pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:31.917021       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:31.958598       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:31.958667       1 controllerserver.go:348] create azure disk(pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f kubernetes.io-created-for-pvc-name:pvc-b8xqq kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:31.958689       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f","csi.storage.k8s.io/pvc/name":"pvc-b8xqq","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f"}}
I1002 14:48:31.999812       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:31.999838       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-519a4e42-8e83-423c-a966-c105f5612432","parameters":{"csi.storage.k8s.io/pv/name":"pvc-519a4e42-8e83-423c-a966-c105f5612432","csi.storage.k8s.io/pvc/name":"pvc-cv22c","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:32.016982       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-519a4e42-8e83-423c-a966-c105f5612432, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-519a4e42-8e83-423c-a966-c105f5612432' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:32.017133       1 controllerserver.go:271] begin to create azure disk(pvc-519a4e42-8e83-423c-a966-c105f5612432) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:32.017190       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-519a4e42-8e83-423c-a966-c105f5612432 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.018462       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.018530       1 controllerserver.go:348] create azure disk(pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49 kubernetes.io-created-for-pvc-name:pvc-r8rlz kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:32.018551       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49","csi.storage.k8s.io/pvc/name":"pvc-r8rlz","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49"}}
I1002 14:48:32.055076       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-8f21becf-3e62-48ca-977c-2514cf9f135f StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.055146       1 controllerserver.go:348] create azure disk(pvc-8f21becf-3e62-48ca-977c-2514cf9f135f) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-8f21becf-3e62-48ca-977c-2514cf9f135f kubernetes.io-created-for-pvc-name:pvc-c586c kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:32.055167       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-8f21becf-3e62-48ca-977c-2514cf9f135f","csi.storage.k8s.io/pvc/name":"pvc-c586c","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8f21becf-3e62-48ca-977c-2514cf9f135f"}}
I1002 14:48:32.104343       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:32.104365       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844","parameters":{"csi.storage.k8s.io/pv/name":"pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844","csi.storage.k8s.io/pvc/name":"pvc-hd6pv","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:32.121484       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:32.121568       1 controllerserver.go:271] begin to create azure disk(pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:32.121582       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.160750       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a414f456-3b4d-492f-9c71-72963857ef6a
I1002 14:48:32.160774       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a414f456-3b4d-492f-9c71-72963857ef6a) returned with <nil>
I1002 14:48:32.160806       1 utils.go:102] GRPC response: {}
I1002 14:48:32.199625       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:32.199643       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362","parameters":{"csi.storage.k8s.io/pv/name":"pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362","csi.storage.k8s.io/pvc/name":"pvc-jvvml","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:32.226826       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:32.226914       1 controllerserver.go:271] begin to create azure disk(pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:32.226941       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.281122       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-9015e829-c629-45b5-89bc-e6defff725a9 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.281165       1 controllerserver.go:348] create azure disk(pvc-9015e829-c629-45b5-89bc-e6defff725a9) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-9015e829-c629-45b5-89bc-e6defff725a9 kubernetes.io-created-for-pvc-name:pvc-cpzk9 kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:32.281189       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9015e829-c629-45b5-89bc-e6defff725a9","csi.storage.k8s.io/pvc/name":"pvc-cpzk9","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9"}}
I1002 14:48:32.380389       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.380438       1 controllerserver.go:348] create azure disk(pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f kubernetes.io-created-for-pvc-name:pvc-8j8n7 kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:32.380465       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f","csi.storage.k8s.io/pvc/name":"pvc-8j8n7","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f"}}
I1002 14:48:32.453985       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.454048       1 controllerserver.go:348] create azure disk(pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d kubernetes.io-created-for-pvc-name:pvc-czkqn kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:32.454074       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d","csi.storage.k8s.io/pvc/name":"pvc-czkqn","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d"}}
I1002 14:48:32.549459       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:32.549483       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd","parameters":{"csi.storage.k8s.io/pv/name":"pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd","csi.storage.k8s.io/pvc/name":"pvc-trg9k","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:32.566852       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:32.566942       1 controllerserver.go:271] begin to create azure disk(pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:32.566958       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.571538       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.571573       1 controllerserver.go:348] create azure disk(pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9 kubernetes.io-created-for-pvc-name:pvc-8qnmx kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:32.571594       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9","csi.storage.k8s.io/pvc/name":"pvc-8qnmx","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9"}}
I1002 14:48:32.580179       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.580214       1 controllerserver.go:348] create azure disk(pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17 kubernetes.io-created-for-pvc-name:pvc-5qsvc kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:32.580233       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17","csi.storage.k8s.io/pvc/name":"pvc-5qsvc","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17"}}
I1002 14:48:32.603696       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:32.603723       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-87fdc28f-7998-439a-a602-b5e00b93bf49","parameters":{"csi.storage.k8s.io/pv/name":"pvc-87fdc28f-7998-439a-a602-b5e00b93bf49","csi.storage.k8s.io/pvc/name":"pvc-wbkfb","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:32.620742       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-87fdc28f-7998-439a-a602-b5e00b93bf49, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-87fdc28f-7998-439a-a602-b5e00b93bf49' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:32.620850       1 controllerserver.go:271] begin to create azure disk(pvc-87fdc28f-7998-439a-a602-b5e00b93bf49) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:32.620887       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-87fdc28f-7998-439a-a602-b5e00b93bf49 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.813551       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.813592       1 controllerserver.go:348] create azure disk(pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b kubernetes.io-created-for-pvc-name:pvc-q5gzm kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:32.813616       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b","csi.storage.k8s.io/pvc/name":"pvc-q5gzm","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b"}}
I1002 14:48:32.937787       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-2ada027c-fb64-42ef-9797-f62647ebd42e StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.937830       1 controllerserver.go:348] create azure disk(pvc-2ada027c-fb64-42ef-9797-f62647ebd42e) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-2ada027c-fb64-42ef-9797-f62647ebd42e kubernetes.io-created-for-pvc-name:pvc-mzlpl kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:32.937855       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-2ada027c-fb64-42ef-9797-f62647ebd42e","csi.storage.k8s.io/pvc/name":"pvc-mzlpl","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e"}}
I1002 14:48:32.943496       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.943535       1 controllerserver.go:348] create azure disk(pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606 kubernetes.io-created-for-pvc-name:pvc-lwrht kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:32.943555       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606","csi.storage.k8s.io/pvc/name":"pvc-lwrht","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606"}}
I1002 14:48:32.949480       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:32.949501       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-8a871f5b-3a28-4f35-8be5-9575d1536811","parameters":{"csi.storage.k8s.io/pv/name":"pvc-8a871f5b-3a28-4f35-8be5-9575d1536811","csi.storage.k8s.io/pvc/name":"pvc-sg2rf","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:32.966695       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8a871f5b-3a28-4f35-8be5-9575d1536811, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-8a871f5b-3a28-4f35-8be5-9575d1536811' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:32.966757       1 controllerserver.go:271] begin to create azure disk(pvc-8a871f5b-3a28-4f35-8be5-9575d1536811) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:32.966794       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-8a871f5b-3a28-4f35-8be5-9575d1536811 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:32.998795       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:32.998828       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465","parameters":{"csi.storage.k8s.io/pv/name":"pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465","csi.storage.k8s.io/pvc/name":"pvc-t9slp","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:33.020251       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:33.020366       1 controllerserver.go:271] begin to create azure disk(pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:33.020417       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:33.083682       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-e19708a1-7049-4d60-a087-03a7611a7f3b StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:33.083742       1 controllerserver.go:348] create azure disk(pvc-e19708a1-7049-4d60-a087-03a7611a7f3b) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-e19708a1-7049-4d60-a087-03a7611a7f3b kubernetes.io-created-for-pvc-name:pvc-mwx4q kubernetes.io-created-for-pvc-namespace:volumelimits-4125]) successfully
I1002 14:48:33.083765       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-e19708a1-7049-4d60-a087-03a7611a7f3b","csi.storage.k8s.io/pvc/name":"pvc-mwx4q","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b"}}
I1002 14:48:33.130725       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-9fa24158-b01a-4725-841c-e55266a60e43 StorageAccountType:StandardSSD_LRS Size:5
... skipping 63 lines ...
I1002 14:48:35.508669       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3  true false 0})])
I1002 14:48:35.519881       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1002 14:48:35.519902       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462"}
I1002 14:48:35.519990       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462 from node k8s-agentpool1-13787793-vmss000000
I1002 14:48:35.520002       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:35.520043       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:35.735060       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:48:35.735225       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462" from node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462:pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462]
I1002 14:48:35.735258       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
E1002 14:48:35.777264       1 azure_controller_vmss.go:183] detach azure disk on node(k8s-agentpool1-13787793-vmss000000): disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462:pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462]) not found
I1002 14:48:35.777303       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462:pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462])
I1002 14:48:38.912846       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1002 14:48:38.912874       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855"}
... skipping 2 lines ...
I1002 14:48:38.913041       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:38.913056       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855" from node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855:pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855]
I1002 14:48:38.913085       1 azure_controller_vmss.go:174] azureDisk - detach disk: name "pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855"
I1002 14:48:38.913100       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855:pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855])
I1002 14:48:38.934099       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:38.934125       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-66789345-6034-4f88-a9d8-ea9b8cea307d","parameters":{"csi.storage.k8s.io/pv/name":"pvc-66789345-6034-4f88-a9d8-ea9b8cea307d","csi.storage.k8s.io/pvc/name":"test.csi.azure.comgx49z","csi.storage.k8s.io/pvc/namespace":"provisioning-25"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:48:38.952155       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-66789345-6034-4f88-a9d8-ea9b8cea307d, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-66789345-6034-4f88-a9d8-ea9b8cea307d' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:38.952232       1 controllerserver.go:271] begin to create azure disk(pvc-66789345-6034-4f88-a9d8-ea9b8cea307d) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:38.952248       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-66789345-6034-4f88-a9d8-ea9b8cea307d StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:39.858289       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1002 14:48:39.858315       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d30a4bd0-a617-497e-836d-c1820e36efa5"}
I1002 14:48:39.858395       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d30a4bd0-a617-497e-836d-c1820e36efa5)
I1002 14:48:40.482259       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 177 lines ...
I1002 14:48:51.564847       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-66789345-6034-4f88-a9d8-ea9b8cea307d" to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:48:51.564874       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:51.564895       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-66789345-6034-4f88-a9d8-ea9b8cea307d" to node "k8s-agentpool1-13787793-vmss000001"
I1002 14:48:52.797955       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1002 14:48:52.797992       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462"}
I1002 14:48:52.798085       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462)
I1002 14:48:52.798102       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462) returned with failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462) since it's in attaching or detaching state
E1002 14:48:52.798142       1 utils.go:100] GRPC error: failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a5f6fa4-8075-4267-a557-6c4ca7fba462) since it's in attaching or detaching state
I1002 14:48:54.192721       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855:pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855]) returned with <nil>
I1002 14:48:54.192777       1 azure_controller_common.go:369] azureDisk - detach disk(pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855) succeeded
I1002 14:48:54.192801       1 controllerserver.go:533] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-4368ff35-cdda-428b-8df2-cb0faf7e0855 from node k8s-agentpool1-13787793-vmss000001 successfully
I1002 14:48:54.192814       1 utils.go:102] GRPC response: {}
I1002 14:48:54.192889       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4" from node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4:pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4 /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b:pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b]
I1002 14:48:54.192916       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:48:54.270888       1 azure_controller_vmss.go:174] azureDisk - detach disk: name "pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4"
I1002 14:48:54.270963       1 azure_controller_vmss.go:174] azureDisk - detach disk: name "pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b"
I1002 14:48:54.270977       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4:pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4 /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b:pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b])
I1002 14:48:54.933860       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:48:54.933907       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf","parameters":{"csi.storage.k8s.io/pv/name":"pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf","csi.storage.k8s.io/pvc/name":"test.csi.azure.com5zljw","csi.storage.k8s.io/pvc/namespace":"volume-2899"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":7}}]}
I1002 14:48:54.951084       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:48:54.951196       1 controllerserver.go:271] begin to create azure disk(pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:48:54.951214       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf StorageAccountType:StandardSSD_LRS Size:5
I1002 14:48:55.876866       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3" attached to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:48:55.876914       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3" to node "k8s-agentpool1-13787793-vmss000000" successfully
I1002 14:48:55.876937       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1002 14:48:55.887303       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 9 lines ...
I1002 14:48:56.602297       1 utils.go:102] GRPC response: {}
I1002 14:48:56.602393       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:56.602427       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:48:56.647229       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f" lun 15 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b  true false 27}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-224ff370-4c43-4738-974f-c260f23439dc  true false 24}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2ada027c-fb64-42ef-9797-f62647ebd42e  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702  true false 22}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd  true false 8}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-519a4e42-8e83-423c-a966-c105f5612432:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-519a4e42-8e83-423c-a966-c105f5612432  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844  true false 16}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-87fdc28f-7998-439a-a602-b5e00b93bf49:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-87fdc28f-7998-439a-a602-b5e00b93bf49  true false 26}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8a871f5b-3a28-4f35-8be5-9575d1536811:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8a871f5b-3a28-4f35-8be5-9575d1536811  true false 28}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9  true false 13}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8f21becf-3e62-48ca-977c-2514cf9f135f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8f21becf-3e62-48ca-977c-2514cf9f135f  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9015e829-c629-45b5-89bc-e6defff725a9  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 5}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ad89e320-0715-45d1-a22c-177c4c9109c3  true false 23}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49  true false 29}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-b9075cab-972d-44e5-b5f5-817883de636f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b9075cab-972d-44e5-b5f5-817883de636f  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dd56b38f-7605-45b2-879d-e35a4eec8723  true false 3}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e19708a1-7049-4d60-a087-03a7611a7f3b  true false 4})]
I1002 14:48:56.647325       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:56.647387       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b  true false 27}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-224ff370-4c43-4738-974f-c260f23439dc  true false 24}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2ada027c-fb64-42ef-9797-f62647ebd42e  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702  true false 22}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd  true false 8}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-519a4e42-8e83-423c-a966-c105f5612432:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-519a4e42-8e83-423c-a966-c105f5612432  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844  true false 16}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-87fdc28f-7998-439a-a602-b5e00b93bf49:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-87fdc28f-7998-439a-a602-b5e00b93bf49  true false 26}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8a871f5b-3a28-4f35-8be5-9575d1536811:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8a871f5b-3a28-4f35-8be5-9575d1536811  true false 28}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9  true false 13}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8f21becf-3e62-48ca-977c-2514cf9f135f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8f21becf-3e62-48ca-977c-2514cf9f135f  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9015e829-c629-45b5-89bc-e6defff725a9  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 5}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ad89e320-0715-45d1-a22c-177c4c9109c3  true false 23}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49  true false 29}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-b9075cab-972d-44e5-b5f5-817883de636f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b9075cab-972d-44e5-b5f5-817883de636f  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dd56b38f-7605-45b2-879d-e35a4eec8723  true false 3}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e19708a1-7049-4d60-a087-03a7611a7f3b  true false 4})])
I1002 14:48:56.741492       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:48:56.741522       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:48:56.741664       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b  true false 27}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-224ff370-4c43-4738-974f-c260f23439dc  true false 24}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2ada027c-fb64-42ef-9797-f62647ebd42e  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702  true false 22}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd  true false 8}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-519a4e42-8e83-423c-a966-c105f5612432:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-519a4e42-8e83-423c-a966-c105f5612432  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844  true false 16}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-87fdc28f-7998-439a-a602-b5e00b93bf49:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-87fdc28f-7998-439a-a602-b5e00b93bf49  true false 26}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8a871f5b-3a28-4f35-8be5-9575d1536811:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8a871f5b-3a28-4f35-8be5-9575d1536811  true false 28}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9  true false 13}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8f21becf-3e62-48ca-977c-2514cf9f135f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8f21becf-3e62-48ca-977c-2514cf9f135f  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9015e829-c629-45b5-89bc-e6defff725a9  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 5}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ad89e320-0715-45d1-a22c-177c4c9109c3  true false 23}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49  true false 29}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-b9075cab-972d-44e5-b5f5-817883de636f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b9075cab-972d-44e5-b5f5-817883de636f  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dd56b38f-7605-45b2-879d-e35a4eec8723  true false 3}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e19708a1-7049-4d60-a087-03a7611a7f3b  true false 4})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:48:56.741890       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b  true false 27}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-224ff370-4c43-4738-974f-c260f23439dc  true false 24}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2ada027c-fb64-42ef-9797-f62647ebd42e  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702  true false 22}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd  true false 8}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-519a4e42-8e83-423c-a966-c105f5612432:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-519a4e42-8e83-423c-a966-c105f5612432  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844  true false 16}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-87fdc28f-7998-439a-a602-b5e00b93bf49:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-87fdc28f-7998-439a-a602-b5e00b93bf49  true false 26}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8a871f5b-3a28-4f35-8be5-9575d1536811:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8a871f5b-3a28-4f35-8be5-9575d1536811  true false 28}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9  true false 13}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8f21becf-3e62-48ca-977c-2514cf9f135f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8f21becf-3e62-48ca-977c-2514cf9f135f  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9015e829-c629-45b5-89bc-e6defff725a9  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 5}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ad89e320-0715-45d1-a22c-177c4c9109c3  true false 23}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49  true false 29}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-b9075cab-972d-44e5-b5f5-817883de636f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b9075cab-972d-44e5-b5f5-817883de636f  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dd56b38f-7605-45b2-879d-e35a4eec8723  true false 3}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e19708a1-7049-4d60-a087-03a7611a7f3b  true false 4})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:48:56.741993       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:48:56.742093       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:48:56.742113       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:56.742147       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:48:56.749917       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:48:56.749945       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f","csi.storage.k8s.io/pvc/name":"pvc-b8xqq","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f"}
I1002 14:48:56.771295       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:48:56.771320       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:56.798497       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362) in current disk list(len: 2) nor in diskMap(map[])
E1002 14:48:56.798564       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362) in current disk list(len: 2) nor in diskMap(map[])
I1002 14:48:56.798564       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:48:56.798612       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:56.798630       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f" lun 1 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f  true false 1})]
I1002 14:48:56.798648       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:56.798665       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f  true false 1})])
I1002 14:48:56.805910       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:48:56.805936       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362","csi.storage.k8s.io/pvc/name":"pvc-jvvml","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362"}
I1002 14:48:56.828401       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:48:56.828446       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:56.828485       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:48:56.973702       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:48:56.973989       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:56.974042       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
E1002 14:48:57.052729       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40) in current disk list(len: 3) nor in diskMap(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:0xc000d44680])
E1002 14:48:57.052775       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40) in current disk list(len: 3) nor in diskMap(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:0xc000d44680])
I1002 14:48:57.052832       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.052881       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-87fdc28f-7998-439a-a602-b5e00b93bf49" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-87fdc28f-7998-439a-a602-b5e00b93bf49) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.052949       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-87fdc28f-7998-439a-a602-b5e00b93bf49" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-87fdc28f-7998-439a-a602-b5e00b93bf49) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.052969       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.053007       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.053028       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.053028       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.053063       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.053114       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.053143       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.053166       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-b9075cab-972d-44e5-b5f5-817883de636f" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-b9075cab-972d-44e5-b5f5-817883de636f) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.053183       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-b9075cab-972d-44e5-b5f5-817883de636f" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-b9075cab-972d-44e5-b5f5-817883de636f) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.053203       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.053231       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.053255       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.053279       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.053303       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.053320       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.053337       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.053353       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8f21becf-3e62-48ca-977c-2514cf9f135f" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8f21becf-3e62-48ca-977c-2514cf9f135f) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.053368       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8f21becf-3e62-48ca-977c-2514cf9f135f" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8f21becf-3e62-48ca-977c-2514cf9f135f) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.053379       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.053393       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.053416       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.053427       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.053443       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.053468       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.053479       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.053502       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.053518       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.053536       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.053558       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8a871f5b-3a28-4f35-8be5-9575d1536811" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8a871f5b-3a28-4f35-8be5-9575d1536811) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.053583       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8a871f5b-3a28-4f35-8be5-9575d1536811" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8a871f5b-3a28-4f35-8be5-9575d1536811) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.053602       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.053652       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.053722       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.053755       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.053785       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.053808       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.053806       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.053839       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.053861       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.053891       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.053912       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.053928       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.053971       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.053990       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.054038       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.054063       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.054080       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.054096       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.054106       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.054120       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.054135       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.054145       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.054159       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.054174       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.054186       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.054200       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.054215       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.054225       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.054239       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-519a4e42-8e83-423c-a966-c105f5612432" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-519a4e42-8e83-423c-a966-c105f5612432) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.054254       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-519a4e42-8e83-423c-a966-c105f5612432" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-519a4e42-8e83-423c-a966-c105f5612432) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.054264       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.054277       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.054292       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.054303       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.054317       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.054331       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.054341       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.054355       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.054378       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.054399       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.054416       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.054432       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.054453       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.054469       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.054484       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.054494       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.054510       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.054524       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.054553       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.054568       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.054584       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.054596       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.054620       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.054634       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.054694       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:48:57.054709       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362) in current disk list(len: 3) nor in diskMap(map[])
E1002 14:48:57.054733       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362) in current disk list(len: 3) nor in diskMap(map[])
I1002 14:48:57.063732       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:48:57.063764       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40","csi.storage.k8s.io/pvc/name":"pvc-6n2pm","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40"}
I1002 14:48:57.066770       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:48:57.066792       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-8f21becf-3e62-48ca-977c-2514cf9f135f","csi.storage.k8s.io/pvc/name":"pvc-c586c","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8f21becf-3e62-48ca-977c-2514cf9f135f"}
I1002 14:48:57.080951       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:48:57.080974       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-87fdc28f-7998-439a-a602-b5e00b93bf49","csi.storage.k8s.io/pvc/name":"pvc-wbkfb","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-87fdc28f-7998-439a-a602-b5e00b93bf49"}
... skipping 57 lines ...
I1002 14:48:57.811758       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0","csi.storage.k8s.io/pvc/name":"pvc-5scjq","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0"}
I1002 14:48:57.841870       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:48:57.841910       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:57.841938       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:48:58.009145       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:48:58.009174       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f","csi.storage.k8s.io/pvc/name":"pvc-8j8n7","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f"}
I1002 14:48:58.017578       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40  true false 3})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:48:58.017793       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:58.017861       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:48:58.033468       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:48:58.033504       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:58.064791       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8f21becf-3e62-48ca-977c-2514cf9f135f" lun 14 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b  true false 5}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2ada027c-fb64-42ef-9797-f62647ebd42e  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-87fdc28f-7998-439a-a602-b5e00b93bf49:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-87fdc28f-7998-439a-a602-b5e00b93bf49  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8a871f5b-3a28-4f35-8be5-9575d1536811:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8a871f5b-3a28-4f35-8be5-9575d1536811  true false 4}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8f21becf-3e62-48ca-977c-2514cf9f135f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8f21becf-3e62-48ca-977c-2514cf9f135f  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-b9075cab-972d-44e5-b5f5-817883de636f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b9075cab-972d-44e5-b5f5-817883de636f  true false 8}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0  true false 13})]
I1002 14:48:58.064838       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 11 lines ...
I1002 14:48:58.435309       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:48:58.610477       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:48:58.610503       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-dd56b38f-7605-45b2-879d-e35a4eec8723","csi.storage.k8s.io/pvc/name":"pvc-5c44d","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723"}
I1002 14:48:58.651253       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:48:58.651307       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:58.651424       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:48:58.668569       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b  true false 5}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2ada027c-fb64-42ef-9797-f62647ebd42e  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7bc91f98-5cd8-4eab-96b3-84c8e5b9d844  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-87fdc28f-7998-439a-a602-b5e00b93bf49:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-87fdc28f-7998-439a-a602-b5e00b93bf49  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8a871f5b-3a28-4f35-8be5-9575d1536811:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8a871f5b-3a28-4f35-8be5-9575d1536811  true false 4}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8f21becf-3e62-48ca-977c-2514cf9f135f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8f21becf-3e62-48ca-977c-2514cf9f135f  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-b9075cab-972d-44e5-b5f5-817883de636f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b9075cab-972d-44e5-b5f5-817883de636f  true false 8}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0  true false 13})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:48:58.668712       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:58.668740       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:48:58.716751       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-b9075cab-972d-44e5-b5f5-817883de636f" lun 8 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dd56b38f-7605-45b2-879d-e35a4eec8723  true false 16})]
I1002 14:48:58.716833       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:58.716951       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dd56b38f-7605-45b2-879d-e35a4eec8723  true false 16})])
I1002 14:48:58.810887       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:48:58.810915       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606","csi.storage.k8s.io/pvc/name":"pvc-lwrht","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606"}
I1002 14:48:58.847556       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:48:58.847604       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:58.847650       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:48:58.970108       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ae69f947-5c7b-4826-acc3-0d91b58dda49  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-dd56b38f-7605-45b2-879d-e35a4eec8723  true false 16})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:48:58.970268       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:58.970303       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:48:59.011574       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:48:59.011604       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9","csi.storage.k8s.io/pvc/name":"pvc-8qnmx","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9"}
I1002 14:48:59.038681       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:48:59.038715       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:59.042419       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8a871f5b-3a28-4f35-8be5-9575d1536811" lun 4 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606  true false 19})]
I1002 14:48:59.042457       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:59.042475       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:48:59.042484       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606  true false 19})])
I1002 14:48:59.211403       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:48:59.211432       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702","csi.storage.k8s.io/pvc/name":"pvc-jb4gl","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702"}
I1002 14:48:59.227237       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606  true false 19})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:48:59.227420       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:59.227523       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:48:59.242676       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:48:59.242709       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:59.322658       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d" lun 9 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9  true false 20})]
I1002 14:48:59.322694       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 6 lines ...
I1002 14:48:59.433020       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:48:59.610555       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:48:59.610582       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ad89e320-0715-45d1-a22c-177c4c9109c3","csi.storage.k8s.io/pvc/name":"pvc-sbnzl","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3"}
I1002 14:48:59.643747       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:48:59.643784       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:59.643890       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:48:59.708374       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8e1d94ed-cbbb-4f3c-99d5-3f11dbf5d9e9  true false 20})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:48:59.708549       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:59.708574       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:48:59.753633       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-87fdc28f-7998-439a-a602-b5e00b93bf49" lun 10 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd  true false 22}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ad89e320-0715-45d1-a22c-177c4c9109c3  true false 23})]
I1002 14:48:59.753935       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:59.754020       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd  true false 22}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ad89e320-0715-45d1-a22c-177c4c9109c3  true false 23})])
I1002 14:48:59.810531       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:48:59.810563       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9015e829-c629-45b5-89bc-e6defff725a9","csi.storage.k8s.io/pvc/name":"pvc-cpzk9","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9"}
I1002 14:48:59.833371       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:48:59.833407       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:59.833443       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:48:59.973689       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-39c64e3f-ed08-472f-92bc-0dbc5a5b9702  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-44c5572f-1ed7-4ff9-962d-56fb5db2ddbd  true false 22}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ad89e320-0715-45d1-a22c-177c4c9109c3  true false 23})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:48:59.973907       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:48:59.973940       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:49:00.010339       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:49:00.010365       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17","csi.storage.k8s.io/pvc/name":"pvc-5qsvc","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17"}
I1002 14:49:00.024948       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-1da4de0e-b63b-4c84-8a0f-96bce3dc142b" lun 5 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9015e829-c629-45b5-89bc-e6defff725a9  true false 24})]
I1002 14:49:00.024996       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:00.025049       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9015e829-c629-45b5-89bc-e6defff725a9  true false 24})])
I1002 14:49:00.037057       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:49:00.037103       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:00.037155       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:00.187908       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9015e829-c629-45b5-89bc-e6defff725a9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9015e829-c629-45b5-89bc-e6defff725a9  true false 24})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:49:00.188082       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:00.188112       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:49:00.211026       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:49:00.211048       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-519a4e42-8e83-423c-a966-c105f5612432","csi.storage.k8s.io/pvc/name":"pvc-cv22c","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-519a4e42-8e83-423c-a966-c105f5612432"}
I1002 14:49:00.276360       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-519a4e42-8e83-423c-a966-c105f5612432" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:49:00.276401       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 3 lines ...
I1002 14:49:00.306126       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17  true false 25})])
I1002 14:49:00.411190       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:49:00.411219       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-e19708a1-7049-4d60-a087-03a7611a7f3b","csi.storage.k8s.io/pvc/name":"pvc-mwx4q","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b"}
I1002 14:49:00.435856       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:49:00.435888       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:00.435946       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:00.472966       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17  true false 25})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:49:00.473183       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:00.473221       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:49:00.535054       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-77258b18-4b42-44d5-b8d5-2d5fb8812c5d" lun 11 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-519a4e42-8e83-423c-a966-c105f5612432:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-519a4e42-8e83-423c-a966-c105f5612432  true false 26}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e19708a1-7049-4d60-a087-03a7611a7f3b  true false 27})]
I1002 14:49:00.535100       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:00.535151       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-519a4e42-8e83-423c-a966-c105f5612432:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-519a4e42-8e83-423c-a966-c105f5612432  true false 26}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e19708a1-7049-4d60-a087-03a7611a7f3b  true false 27})])
I1002 14:49:00.610308       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:49:00.610376       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21","csi.storage.k8s.io/pvc/name":"pvc-lff45","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21"}
I1002 14:49:00.641320       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:49:00.641356       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:00.641396       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:00.731662       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-519a4e42-8e83-423c-a966-c105f5612432:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-519a4e42-8e83-423c-a966-c105f5612432  true false 26}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e19708a1-7049-4d60-a087-03a7611a7f3b  true false 27})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:49:00.731828       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:00.731852       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:49:00.810372       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:49:00.810402       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-224ff370-4c43-4738-974f-c260f23439dc","csi.storage.k8s.io/pvc/name":"pvc-5cp44","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc"}
I1002 14:49:00.820672       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742" lun 7 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21  true false 28})]
I1002 14:49:00.820733       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:00.820794       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21  true false 28})])
I1002 14:49:00.835207       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:49:00.835242       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:00.835349       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:00.983395       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ae08470a-25fc-48d2-bd5b-3aef983e7c21  true false 28})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:49:00.983646       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:00.983677       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:49:01.010717       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:49:01.010744       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465","csi.storage.k8s.io/pvc/name":"pvc-t9slp","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465"}
I1002 14:49:01.033523       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:49:01.033559       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:01.035052       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e" lun 12 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-224ff370-4c43-4738-974f-c260f23439dc  true false 29})]
I1002 14:49:01.035089       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:01.035125       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-224ff370-4c43-4738-974f-c260f23439dc  true false 29})])
I1002 14:49:01.035164       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:01.209484       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:49:01.209513       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362","csi.storage.k8s.io/pvc/name":"pvc-jvvml","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362"}
I1002 14:49:01.216672       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-224ff370-4c43-4738-974f-c260f23439dc:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-224ff370-4c43-4738-974f-c260f23439dc  true false 29})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:49:01.217188       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:01.217243       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:49:01.233312       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:49:01.233343       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:01.300312       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ba438ad3-1858-44f7-b5fd-13459ec894f0" lun 13 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465  true false 30})]
I1002 14:49:01.300352       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:01.300391       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465  true false 30})])
I1002 14:49:01.300594       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:01.409331       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:49:01.409361       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9fa24158-b01a-4725-841c-e55266a60e43","csi.storage.k8s.io/pvc/name":"pvc-m2qqs","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43"}
I1002 14:49:01.432764       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:49:01.432816       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:01.432875       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:01.493056       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465  true false 30})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:49:01.493318       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:01.493352       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:49:01.561326       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f" lun 17 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]
I1002 14:49:01.561391       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:01.561442       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})])
I1002 14:49:01.610131       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:49:01.610167       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0","csi.storage.k8s.io/pvc/name":"pvc-jdtg5","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0"}
I1002 14:49:01.633557       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:01.633588       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:49:01.633640       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:49:01.633680       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:49:01.633729       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:49:01.633758       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-197c3a92-e9ad-432e-86b3-e2f5386e331f" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:01.633821       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 11 lines ...
I1002 14:49:01.731805       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-6e7b61af-c0af-455a-9e13-7822dd9fe8c0" to node "k8s-agentpool1-13787793-vmss000000" successfully
I1002 14:49:01.731822       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"15"}}
I1002 14:49:01.731878       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:01.731916       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dd56b38f-7605-45b2-879d-e35a4eec8723" lun 16 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0  true false 31})]
I1002 14:49:01.731932       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:01.731969       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0  true false 31})])
I1002 14:49:01.907600       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0  true false 31})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:49:01.907845       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:01.907893       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:49:01.986271       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606" lun 19 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[]
I1002 14:49:01.986322       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606" attached to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:49:01.986335       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606" to node "k8s-agentpool1-13787793-vmss000000" successfully
I1002 14:49:01.986351       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"19"}}
... skipping 50 lines ...
I1002 14:49:01.987778       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:01.987812       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465" lun 30 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[]
I1002 14:49:01.987826       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465" attached to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:49:01.987836       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6efc5d3-119f-4fbb-8131-5e42c7f72465" to node "k8s-agentpool1-13787793-vmss000000" successfully
I1002 14:49:01.987851       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"30"}}
I1002 14:49:01.987891       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:49:01.987931       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362) in current disk list(len: 32) nor in diskMap(map[])
E1002 14:49:01.987954       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362) in current disk list(len: 32) nor in diskMap(map[])
I1002 14:49:01.987983       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:49:01.988019       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43) in current disk list(len: 32) nor in diskMap(map[])
E1002 14:49:01.988037       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43) in current disk list(len: 32) nor in diskMap(map[])
I1002 14:49:01.988049       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:01.988072       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0" lun 31 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[]
I1002 14:49:01.988113       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0" attached to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:49:01.988123       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-5d964721-85c6-47af-8ba2-5bbbd310beb0" to node "k8s-agentpool1-13787793-vmss000000" successfully
I1002 14:49:01.988152       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"31"}}
I1002 14:49:02.009818       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 49 lines ...
I1002 14:49:05.233922       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:05.233979       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:05.233998       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:05.234100       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]
I1002 14:49:05.234130       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:05.234180       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})])
I1002 14:49:05.300357       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:05.300420       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:49:05.300500       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:49:05.300562       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:49:05.300616       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:49:05.300668       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:05.409531       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1002 14:49:05.434909       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:49:05.502088       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:05.502154       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:05.502175       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})]
I1002 14:49:05.502227       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:05.502274       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})])
I1002 14:49:05.582582       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:05.582610       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:49:05.582658       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:49:05.582686       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:49:05.582717       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:49:05.582791       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:05.609055       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 10 lines ...
I1002 14:49:07.286055       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:07.286104       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:07.286131       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:07.286146       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]
I1002 14:49:07.286171       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:07.286207       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})])
I1002 14:49:07.348572       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:07.348601       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:49:07.348683       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:49:07.348729       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:49:07.348785       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:49:07.348815       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:08.613394       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 6 lines ...
I1002 14:49:09.432898       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:49:09.504379       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:09.504443       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:09.504467       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})]
I1002 14:49:09.504484       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:09.504571       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})])
I1002 14:49:09.576103       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:09.576148       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:49:09.576219       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:49:09.576278       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:49:09.576333       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:49:09.576400       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:11.116326       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:49:11.116355       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-cc756372-10e6-4c4a-9a17-148d8582402b","parameters":{"csi.storage.k8s.io/pv/name":"pvc-cc756372-10e6-4c4a-9a17-148d8582402b","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-8ms2w-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-6615"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:49:11.138142       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:49:11.138216       1 controllerserver.go:271] begin to create azure disk(pvc-cc756372-10e6-4c4a-9a17-148d8582402b) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:49:11.138257       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-cc756372-10e6-4c4a-9a17-148d8582402b StorageAccountType:StandardSSD_LRS Size:5
I1002 14:49:13.133085       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40" attached to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:49:13.133130       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40" to node "k8s-agentpool1-13787793-vmss000000" successfully
I1002 14:49:13.133168       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"3"}}
I1002 14:49:13.143236       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 26 lines ...
I1002 14:49:14.269912       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:14.269949       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:14.269973       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:14.269985       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]
I1002 14:49:14.270049       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:14.270092       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})])
I1002 14:49:14.333200       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:14.333228       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:49:14.333275       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:49:14.333323       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:49:14.333376       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:49:14.333406       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:14.343160       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1002 14:49:14.366388       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:49:14.429541       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:14.429586       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:14.429608       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]
I1002 14:49:14.429625       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:14.429673       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})])
I1002 14:49:14.493521       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:14.493547       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:49:14.493599       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:49:14.493628       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:49:14.493678       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:49:14.493730       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:14.500111       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 10 lines ...
I1002 14:49:14.607508       1 azure_controller_common.go:369] azureDisk - detach disk(pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4) succeeded
I1002 14:49:14.607528       1 controllerserver.go:533] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-75aaaf11-e89e-4b37-b3d4-a5f3440b50e4 from node k8s-agentpool1-13787793-vmss000001 successfully
I1002 14:49:14.607557       1 utils.go:102] GRPC response: {}
I1002 14:49:14.607694       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b" from node "k8s-agentpool1-13787793-vmss000001", diskMap: map[]
I1002 14:49:14.607750       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:14.607769       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:49:14.632535       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:14.632560       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:49:14.632598       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:49:14.632646       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
I1002 14:49:14.659270       1 azure_controller_common.go:369] azureDisk - detach disk(pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b) succeeded
I1002 14:49:14.659299       1 controllerserver.go:533] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-dbf58ad7-19d3-4bde-8ae2-e950e8dfd48b from node k8s-agentpool1-13787793-vmss000001 successfully
I1002 14:49:14.659330       1 utils.go:102] GRPC response: {}
E1002 14:49:14.659383       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:14.659412       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1002 14:49:14.659423       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:14.659433       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-66789345-6034-4f88-a9d8-ea9b8cea307d" lun 0 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-66789345-6034-4f88-a9d8-ea9b8cea307d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-66789345-6034-4f88-a9d8-ea9b8cea307d  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf  true false 1})]
I1002 14:49:14.659463       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:14.659478       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-66789345-6034-4f88-a9d8-ea9b8cea307d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-66789345-6034-4f88-a9d8-ea9b8cea307d  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf  true false 1})])
I1002 14:49:14.854941       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-66789345-6034-4f88-a9d8-ea9b8cea307d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-66789345-6034-4f88-a9d8-ea9b8cea307d  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:49:14.855129       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:14.855162       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:49:14.922287       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf" lun 1 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[]
I1002 14:49:14.922334       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:49:14.922346       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 14:49:14.922364       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
... skipping 3 lines ...
I1002 14:49:15.380157       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:15.380233       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:15.380251       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:15.380280       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]
I1002 14:49:15.380303       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:15.380351       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})])
I1002 14:49:15.443858       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:15.443900       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:49:15.444008       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:49:15.444071       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:49:15.444173       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:49:15.444223       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:15.611440       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:49:15.611474       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9fa24158-b01a-4725-841c-e55266a60e43","csi.storage.k8s.io/pvc/name":"pvc-m2qqs","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43"}
I1002 14:49:15.635218       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:49:15.635243       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-4da74381-d01b-4234-9dad-1d24b1920074","parameters":{"csi.storage.k8s.io/pv/name":"pvc-4da74381-d01b-4234-9dad-1d24b1920074","csi.storage.k8s.io/pvc/name":"test.csi.azure.com2plmb","csi.storage.k8s.io/pvc/namespace":"provisioning-3432"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:49:15.658292       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-4da74381-d01b-4234-9dad-1d24b1920074, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-4da74381-d01b-4234-9dad-1d24b1920074' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:49:15.658433       1 controllerserver.go:271] begin to create azure disk(pvc-4da74381-d01b-4234-9dad-1d24b1920074) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:49:15.658466       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-4da74381-d01b-4234-9dad-1d24b1920074 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:49:15.672349       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:49:15.672382       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:15.672403       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:49:15.728724       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:15.728766       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:15.728787       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]
I1002 14:49:15.728803       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:15.728838       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})])
I1002 14:49:15.814435       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:15.814478       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:49:15.814543       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:49:15.814582       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:49:15.814636       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:49:15.814684       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:17.215945       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-c6e36566-ab36-4660-90b8-17c1cc0ed01f" attached to node "k8s-agentpool1-13787793-vmss000000".
... skipping 16 lines ...
I1002 14:49:20.352255       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:20.352288       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-4da74381-d01b-4234-9dad-1d24b1920074" to node "k8s-agentpool1-13787793-vmss000001"
I1002 14:49:20.352309       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:20.352322       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-4da74381-d01b-4234-9dad-1d24b1920074" lun 2 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4da74381-d01b-4234-9dad-1d24b1920074:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4da74381-d01b-4234-9dad-1d24b1920074  true false 2})]
I1002 14:49:20.352373       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:20.352418       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4da74381-d01b-4234-9dad-1d24b1920074:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4da74381-d01b-4234-9dad-1d24b1920074  true false 2})])
I1002 14:49:20.527917       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-4da74381-d01b-4234-9dad-1d24b1920074:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4da74381-d01b-4234-9dad-1d24b1920074  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:49:23.451692       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:49:23.451742       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-cc756372-10e6-4c4a-9a17-148d8582402b","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-8ms2w-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-6615","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b"}
I1002 14:49:23.477446       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:49:23.477511       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:23.477580       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:23.477615       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:23.477628       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]
I1002 14:49:23.477665       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:23.477715       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})])
I1002 14:49:23.553867       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:23.553894       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:49:23.553982       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:49:23.554087       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:49:23.554130       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:49:23.554155       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:25.585812       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1002 14:49:25.611348       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:49:25.693252       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:25.693295       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:25.693318       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})]
I1002 14:49:25.693335       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:25.693367       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})])
I1002 14:49:25.768289       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:25.768318       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:49:25.768424       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:49:25.768468       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:49:25.768505       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:49:25.768545       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:31.822084       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1002 14:49:31.851715       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:49:31.919618       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:31.919685       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:31.919707       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]
I1002 14:49:31.919748       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:31.919825       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})])
I1002 14:49:31.982564       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:31.982586       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:49:31.982647       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:49:31.982700       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:49:31.982738       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:49:31.982760       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:39.565189       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1002 14:49:39.595842       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:49:39.660033       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:39.660101       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:39.660140       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]
I1002 14:49:39.660158       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:39.660217       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})])
I1002 14:49:39.719217       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:39.719284       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:49:39.719335       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:49:39.719363       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:49:39.719400       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:49:39.719434       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:41.213738       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7d2c6edb-548f-40e6-9ef4-c895f464b742" attached to node "k8s-agentpool1-13787793-vmss000000".
... skipping 106 lines ...
I1002 14:49:57.802553       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:57.802633       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:49:57.802649       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:57.802679       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})]
I1002 14:49:57.802719       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:49:57.802752       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})])
I1002 14:49:57.865495       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:57.865555       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:49:57.865628       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:49:57.865684       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:49:57.865724       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:49:57.865764       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:49:59.274724       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:49:59.274766       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033","parameters":{"csi.storage.k8s.io/pv/name":"pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033","csi.storage.k8s.io/pvc/name":"test.csi.azure.com58zt6","csi.storage.k8s.io/pvc/namespace":"provisioning-1836"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:49:59.302504       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:49:59.302578       1 controllerserver.go:271] begin to create azure disk(pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:49:59.302591       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:49:59.442268       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1002 14:49:59.442296       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf"}
I1002 14:49:59.442402       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf from node k8s-agentpool1-13787793-vmss000001
I1002 14:49:59.442415       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 29 lines ...
I1002 14:50:04.103324       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]
I1002 14:50:04.103340       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:50:04.103370       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})])
I1002 14:50:04.155107       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033" to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:50:04.155140       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:50:04.155168       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033" to node "k8s-agentpool1-13787793-vmss000001"
I1002 14:50:04.163535       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:50:04.163570       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:50:04.163646       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:50:04.163688       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:50:04.163728       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:50:04.163790       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:50:11.730527       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1002 14:50:11.754805       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:50:11.837121       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:50:11.837167       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:50:11.837555       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]
I1002 14:50:11.837591       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:50:11.837627       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})])
I1002 14:50:11.902177       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:50:11.902214       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:50:11.902271       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:50:11.902316       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:50:11.902353       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:50:11.902379       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:50:14.719480       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf:pvc-fdb77614-7d7d-4586-a5ce-2adeece19abf]) returned with <nil>
... skipping 28 lines ...
I1002 14:50:35.190100       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:50:35.190123       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-66789345-6034-4f88-a9d8-ea9b8cea307d" from node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-66789345-6034-4f88-a9d8-ea9b8cea307d:pvc-66789345-6034-4f88-a9d8-ea9b8cea307d]
E1002 14:50:35.190182       1 azure_controller_vmss.go:183] detach azure disk on node(k8s-agentpool1-13787793-vmss000001): disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-66789345-6034-4f88-a9d8-ea9b8cea307d:pvc-66789345-6034-4f88-a9d8-ea9b8cea307d]) not found
I1002 14:50:35.190201       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-66789345-6034-4f88-a9d8-ea9b8cea307d:pvc-66789345-6034-4f88-a9d8-ea9b8cea307d])
I1002 14:50:35.226926       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:50:35.226956       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-180001d7-70c1-4adf-919f-7c15369f0909","parameters":{"csi.storage.k8s.io/pv/name":"pvc-180001d7-70c1-4adf-919f-7c15369f0909","csi.storage.k8s.io/pvc/name":"test.csi.azure.comfjgt6","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-6409"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:50:35.253536       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:50:35.253624       1 controllerserver.go:271] begin to create azure disk(pvc-180001d7-70c1-4adf-919f-7c15369f0909) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:50:35.253638       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-180001d7-70c1-4adf-919f-7c15369f0909 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:50:37.556611       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-180001d7-70c1-4adf-919f-7c15369f0909 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:50:37.556673       1 controllerserver.go:348] create azure disk(pvc-180001d7-70c1-4adf-919f-7c15369f0909) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-180001d7-70c1-4adf-919f-7c15369f0909 kubernetes.io-created-for-pvc-name:test.csi.azure.comfjgt6 kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-6409]) successfully
I1002 14:50:37.556716       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-180001d7-70c1-4adf-919f-7c15369f0909","csi.storage.k8s.io/pvc/name":"test.csi.azure.comfjgt6","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-6409","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909"}}
I1002 14:50:39.762035       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
... skipping 15 lines ...
I1002 14:50:40.379272       1 utils.go:102] GRPC response: {}
I1002 14:50:40.379293       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:50:40.379319       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:50:40.441020       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033" lun 0 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-180001d7-70c1-4adf-919f-7c15369f0909  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033  true false 0})]
I1002 14:50:40.441065       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:50:40.441087       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-180001d7-70c1-4adf-919f-7c15369f0909  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033  true false 0})])
I1002 14:50:40.608296       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-180001d7-70c1-4adf-919f-7c15369f0909  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:50:40.608504       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:50:40.608534       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:50:40.667253       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909" lun 1 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[]
I1002 14:50:40.667297       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:50:40.667310       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 14:50:40.667327       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
... skipping 17 lines ...
I1002 14:51:05.077066       1 utils.go:102] GRPC response: {}
I1002 14:51:05.077177       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:51:05.077192       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:51:05.159426       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" lun 2 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 2})]
I1002 14:51:05.159472       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:51:05.159525       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 2})])
I1002 14:51:05.354477       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:51:05.757512       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:51:05.757550       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 14:51:05.757568       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1002 14:51:05.764703       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:51:05.764740       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033","csi.storage.k8s.io/pvc/name":"test.csi.azure.com58zt6","csi.storage.k8s.io/pvc/namespace":"provisioning-1836","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033"}
I1002 14:51:05.790586       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033" to node "k8s-agentpool1-13787793-vmss000001".
... skipping 6 lines ...
I1002 14:51:07.409420       1 utils.go:102] GRPC response: {}
I1002 14:51:07.482391       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:51:07.482418       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790","parameters":{"csi.storage.k8s.io/pv/name":"pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790","csi.storage.k8s.io/pvc/name":"test.csi.azure.com8g7xk","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-822"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:51:07.490256       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1002 14:51:07.490284       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-03f3f456-8874-43a6-8c39-e4777459d5ae"}
I1002 14:51:07.490362       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-03f3f456-8874-43a6-8c39-e4777459d5ae)
I1002 14:51:07.503289       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:51:07.503373       1 controllerserver.go:271] begin to create azure disk(pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:51:07.503393       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:51:08.171996       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:51:08.172023       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9fa24158-b01a-4725-841c-e55266a60e43","csi.storage.k8s.io/pvc/name":"pvc-m2qqs","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43"}
I1002 14:51:08.199675       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:51:08.199707       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:51:08.199730       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:51:08.253406       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:51:08.253452       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:51:08.253475       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]
I1002 14:51:08.253493       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:51:08.253527       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})])
I1002 14:51:08.322348       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:51:08.322379       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:51:08.322449       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:51:08.322483       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:51:08.322527       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:51:08.322561       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:51:09.797980       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790 StorageAccountType:StandardSSD_LRS Size:5
... skipping 14 lines ...
I1002 14:51:12.290313       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790" to node "k8s-agentpool1-13787793-vmss000001"
I1002 14:51:12.726711       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-03f3f456-8874-43a6-8c39-e4777459d5ae
I1002 14:51:12.726758       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-03f3f456-8874-43a6-8c39-e4777459d5ae) returned with <nil>
I1002 14:51:12.726775       1 utils.go:102] GRPC response: {}
I1002 14:51:15.229905       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:51:15.229931       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379","parameters":{"csi.storage.k8s.io/pv/name":"pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379","csi.storage.k8s.io/pvc/name":"pvc-46l7c","csi.storage.k8s.io/pvc/namespace":"provisioning-1685"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":7}}]}
I1002 14:51:15.396603       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:51:15.396676       1 controllerserver.go:271] begin to create azure disk(pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:51:15.396694       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:51:15.910665       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:51:15.910693       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-cc756372-10e6-4c4a-9a17-148d8582402b","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-8ms2w-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-6615","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b"}
I1002 14:51:15.936844       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:51:15.936885       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:51:15.936909       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:51:16.004192       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:51:16.004271       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:51:16.004295       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]
I1002 14:51:16.004312       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:51:16.004345       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})])
I1002 14:51:16.079800       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:51:16.079843       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:51:16.079937       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:51:16.079968       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:51:16.080014       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:51:16.080039       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:51:17.699077       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379 StorageAccountType:StandardSSD_LRS Size:5
... skipping 15 lines ...
I1002 14:51:30.132294       1 utils.go:102] GRPC response: {}
I1002 14:51:30.132377       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:51:30.132404       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:51:30.207691       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790" lun 2 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379  true false 1})]
I1002 14:51:30.207748       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:51:30.207771       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379  true false 1})])
I1002 14:51:30.402451       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:51:30.402691       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033" from node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033:pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033]
I1002 14:51:30.402736       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:51:30.452926       1 azure_controller_vmss.go:174] azureDisk - detach disk: name "pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033"
I1002 14:51:30.452962       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033:pvc-a6a7314f-ed98-40a8-8f9b-bbbd521b9033])
I1002 14:51:30.486263       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" attached to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:51:30.486315       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-904aabae-0b7c-4fcf-a6d3-6c432f6a1362" to node "k8s-agentpool1-13787793-vmss000000" successfully
... skipping 19 lines ...
I1002 14:51:45.654518       1 utils.go:102] GRPC response: {}
I1002 14:51:45.654598       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:51:45.654622       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:51:45.722781       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379" lun 1 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-180001d7-70c1-4adf-919f-7c15369f0909  true false 0})]
I1002 14:51:45.722831       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:51:45.722858       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-180001d7-70c1-4adf-919f-7c15369f0909  true false 0})])
I1002 14:51:45.947734       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-180001d7-70c1-4adf-919f-7c15369f0909  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:51:45.947944       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:51:45.947979       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:51:46.005325       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909" lun 0 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[]
I1002 14:51:46.005379       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:51:46.005426       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 14:51:46.005443       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
... skipping 19 lines ...
I1002 14:51:56.131196       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:51:56.131218       1 azure_controller_common.go:457] azureDisk - find disk: lun 1 name "pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379"
I1002 14:51:56.131230       1 controllerserver.go:455] Attach operation is successful. volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379" is already attached to node "k8s-agentpool1-13787793-vmss000001" at lun 1.
I1002 14:51:56.131248       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
I1002 14:52:00.524185       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:52:00.524213       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97","parameters":{"csi.storage.k8s.io/pv/name":"pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97","csi.storage.k8s.io/pvc/name":"test.csi.azure.comcrkrb","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-8111"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:52:00.543546       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:52:00.543639       1 controllerserver.go:271] begin to create azure disk(pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:52:00.543657       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:52:02.995009       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:52:02.995077       1 controllerserver.go:348] create azure disk(pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97 kubernetes.io-created-for-pvc-name:test.csi.azure.comcrkrb kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-8111]) successfully
I1002 14:52:02.995126       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97","csi.storage.k8s.io/pvc/name":"test.csi.azure.comcrkrb","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-8111","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97"}}
I1002 14:52:05.242089       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 2 lines ...
I1002 14:52:05.267665       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:52:05.267688       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97" to node "k8s-agentpool1-13787793-vmss000001"
I1002 14:52:05.267720       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:52:05.267750       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97" lun 3 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97  true false 3})]
I1002 14:52:05.267789       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:52:05.267807       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97  true false 3})])
I1002 14:52:05.418605       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97  true false 3})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:52:15.495369       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:52:15.495449       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 14:52:15.495480       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"3"}}
I1002 14:52:30.382246       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1002 14:52:30.382274       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909"}
I1002 14:52:30.382392       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909 from node k8s-agentpool1-13787793-vmss000001
... skipping 10 lines ...
I1002 14:52:34.595627       1 utils.go:96] GRPC request: {"name":"snapshot-b2f7296f-0ed1-44cd-964a-d071a60b3e9e","source_volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379"}
I1002 14:52:34.595786       1 controllerserver.go:909] begin to create snapshot(snapshot-b2f7296f-0ed1-44cd-964a-d071a60b3e9e, incremental: true) under rg(kubetest-xak8dfvt)
I1002 14:52:36.918473       1 controllerserver.go:918] create snapshot(snapshot-b2f7296f-0ed1-44cd-964a-d071a60b3e9e) under rg(kubetest-xak8dfvt) successfully
I1002 14:52:36.941060       1 utils.go:102] GRPC response: {"snapshot":{"creation_time":{"nanos":774258700,"seconds":1633186354},"ready_to_use":true,"size_bytes":5368709120,"snapshot_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/snapshots/snapshot-b2f7296f-0ed1-44cd-964a-d071a60b3e9e","source_volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379"}}
I1002 14:52:39.329910       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:52:39.329943       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-9043bc85-aa94-4ecb-b3ce-6bc2ef8fb2e7","parameters":{"csi.storage.k8s.io/pv/name":"pvc-9043bc85-aa94-4ecb-b3ce-6bc2ef8fb2e7","csi.storage.k8s.io/pvc/name":"pvc-jpbb8","csi.storage.k8s.io/pvc/namespace":"provisioning-1685"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":7}}],"volume_content_source":{"Type":{"Snapshot":{"snapshot_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/snapshots/snapshot-b2f7296f-0ed1-44cd-964a-d071a60b3e9e"}}}}
I1002 14:52:39.350376       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9043bc85-aa94-4ecb-b3ce-6bc2ef8fb2e7, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-9043bc85-aa94-4ecb-b3ce-6bc2ef8fb2e7' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:52:39.350463       1 controllerserver.go:271] begin to create azure disk(pvc-9043bc85-aa94-4ecb-b3ce-6bc2ef8fb2e7) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:52:39.350483       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-9043bc85-aa94-4ecb-b3ce-6bc2ef8fb2e7 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:52:40.501653       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1002 14:52:40.501685       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790"}
I1002 14:52:40.501837       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-37c8ec3d-a210-4b75-9007-2b9ea6bc5790 from node k8s-agentpool1-13787793-vmss000001
I1002 14:52:40.501849       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 39 lines ...
I1002 14:52:56.021348       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9043bc85-aa94-4ecb-b3ce-6bc2ef8fb2e7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9043bc85-aa94-4ecb-b3ce-6bc2ef8fb2e7  true false 0})])
I1002 14:52:56.032417       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1002 14:52:56.032439       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379"}
I1002 14:52:56.032527       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379 from node k8s-agentpool1-13787793-vmss000001
I1002 14:52:56.032536       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:52:56.032570       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:52:56.182947       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9043bc85-aa94-4ecb-b3ce-6bc2ef8fb2e7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9043bc85-aa94-4ecb-b3ce-6bc2ef8fb2e7  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:52:56.183108       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379" from node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379:pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379]
I1002 14:52:56.183171       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
E1002 14:52:56.233705       1 azure_controller_vmss.go:183] detach azure disk on node(k8s-agentpool1-13787793-vmss000001): disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379:pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379]) not found
I1002 14:52:56.233741       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379:pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379])
I1002 14:52:59.018923       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909
I1002 14:52:59.018962       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-180001d7-70c1-4adf-919f-7c15369f0909) returned with <nil>
... skipping 13 lines ...
I1002 14:53:06.472396       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97" from node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97:pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97]
I1002 14:53:06.472431       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:53:06.537443       1 azure_controller_vmss.go:174] azureDisk - detach disk: name "pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97"
I1002 14:53:06.537493       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97:pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97])
I1002 14:53:06.834841       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:53:06.834870       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-e3be55d9-4582-4bea-8612-93bc36c67b45","parameters":{"csi.storage.k8s.io/pv/name":"pvc-e3be55d9-4582-4bea-8612-93bc36c67b45","csi.storage.k8s.io/pvc/name":"test.csi.azure.comzrft6","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-8136"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:53:06.854889       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e3be55d9-4582-4bea-8612-93bc36c67b45, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-e3be55d9-4582-4bea-8612-93bc36c67b45' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:53:06.855023       1 controllerserver.go:271] begin to create azure disk(pvc-e3be55d9-4582-4bea-8612-93bc36c67b45) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:53:06.855039       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-e3be55d9-4582-4bea-8612-93bc36c67b45 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:53:09.173043       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-e3be55d9-4582-4bea-8612-93bc36c67b45 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:53:09.173089       1 controllerserver.go:348] create azure disk(pvc-e3be55d9-4582-4bea-8612-93bc36c67b45) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-e3be55d9-4582-4bea-8612-93bc36c67b45 kubernetes.io-created-for-pvc-name:test.csi.azure.comzrft6 kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-8136]) successfully
I1002 14:53:09.173114       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-e3be55d9-4582-4bea-8612-93bc36c67b45","csi.storage.k8s.io/pvc/name":"test.csi.azure.comzrft6","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-8136","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e3be55d9-4582-4bea-8612-93bc36c67b45"}}
I1002 14:53:10.779316       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 13 lines ...
I1002 14:53:16.357681       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:53:16.357749       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:53:16.357771       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:53:16.357808       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]
I1002 14:53:16.357830       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:53:16.357897       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})])
I1002 14:53:16.451641       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:53:16.451672       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:53:16.451726       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:53:16.451756       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:53:16.451793       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:53:16.451826       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:53:21.794446       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97:pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97]) returned with <nil>
... skipping 2 lines ...
I1002 14:53:21.794527       1 utils.go:102] GRPC response: {}
I1002 14:53:21.794609       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:53:21.794675       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:53:21.878800       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e3be55d9-4582-4bea-8612-93bc36c67b45" lun 1 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-e3be55d9-4582-4bea-8612-93bc36c67b45:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e3be55d9-4582-4bea-8612-93bc36c67b45  true false 1})]
I1002 14:53:21.878864       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:53:21.878883       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-e3be55d9-4582-4bea-8612-93bc36c67b45:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e3be55d9-4582-4bea-8612-93bc36c67b45  true false 1})])
I1002 14:53:22.049523       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-e3be55d9-4582-4bea-8612-93bc36c67b45:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e3be55d9-4582-4bea-8612-93bc36c67b45  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:53:24.088831       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:53:24.088861       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-cc756372-10e6-4c4a-9a17-148d8582402b","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-8ms2w-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-6615","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b"}
I1002 14:53:24.128931       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:53:24.128987       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:53:24.129080       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:53:24.129117       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:53:24.129147       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]
I1002 14:53:24.129182       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:53:24.129214       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})])
I1002 14:53:24.191386       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:53:24.191414       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:53:24.191485       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:53:24.191550       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:53:24.191584       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:53:24.191633       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:53:27.491097       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 10 lines ...
I1002 14:53:30.901219       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9043bc85-aa94-4ecb-b3ce-6bc2ef8fb2e7:pvc-9043bc85-aa94-4ecb-b3ce-6bc2ef8fb2e7])
I1002 14:53:32.723190       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97
I1002 14:53:32.723227       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9c9cc9a9-8b39-407e-9aa6-856f72288e97) returned with <nil>
I1002 14:53:32.723245       1 utils.go:102] GRPC response: {}
I1002 14:53:41.326191       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:53:41.326220       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-3f580930-36fc-4953-a1c9-e101f467a95f","parameters":{"csi.storage.k8s.io/pv/name":"pvc-3f580930-36fc-4953-a1c9-e101f467a95f","csi.storage.k8s.io/pvc/name":"inline-volume-tester-pqv7k-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-8068"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:53:41.346546       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-3f580930-36fc-4953-a1c9-e101f467a95f, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-3f580930-36fc-4953-a1c9-e101f467a95f' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:53:41.346632       1 controllerserver.go:271] begin to create azure disk(pvc-3f580930-36fc-4953-a1c9-e101f467a95f) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:53:41.346651       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-3f580930-36fc-4953-a1c9-e101f467a95f StorageAccountType:StandardSSD_LRS Size:5
I1002 14:53:43.684411       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-3f580930-36fc-4953-a1c9-e101f467a95f StorageAccountType:StandardSSD_LRS Size:5
I1002 14:53:43.684477       1 controllerserver.go:348] create azure disk(pvc-3f580930-36fc-4953-a1c9-e101f467a95f) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-3f580930-36fc-4953-a1c9-e101f467a95f kubernetes.io-created-for-pvc-name:inline-volume-tester-pqv7k-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-8068]) successfully
I1002 14:53:43.684519       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-3f580930-36fc-4953-a1c9-e101f467a95f","csi.storage.k8s.io/pvc/name":"inline-volume-tester-pqv7k-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-8068","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-3f580930-36fc-4953-a1c9-e101f467a95f"}}
I1002 14:53:44.431558       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 7 lines ...
I1002 14:53:51.203716       1 utils.go:102] GRPC response: {}
I1002 14:53:51.203795       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:53:51.203854       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:53:51.282355       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-3f580930-36fc-4953-a1c9-e101f467a95f" lun 0 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-3f580930-36fc-4953-a1c9-e101f467a95f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3f580930-36fc-4953-a1c9-e101f467a95f  true false 0})]
I1002 14:53:51.282402       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:53:51.282424       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-3f580930-36fc-4953-a1c9-e101f467a95f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3f580930-36fc-4953-a1c9-e101f467a95f  true false 0})])
I1002 14:53:51.437662       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-3f580930-36fc-4953-a1c9-e101f467a95f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3f580930-36fc-4953-a1c9-e101f467a95f  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:53:52.347511       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e3be55d9-4582-4bea-8612-93bc36c67b45" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:53:52.347551       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e3be55d9-4582-4bea-8612-93bc36c67b45" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 14:53:52.347604       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
I1002 14:53:52.356211       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:53:52.356232       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-e3be55d9-4582-4bea-8612-93bc36c67b45","csi.storage.k8s.io/pvc/name":"test.csi.azure.comzrft6","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-8136","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e3be55d9-4582-4bea-8612-93bc36c67b45"}
I1002 14:53:52.380948       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e3be55d9-4582-4bea-8612-93bc36c67b45" to node "k8s-agentpool1-13787793-vmss000001".
... skipping 28 lines ...
I1002 14:54:13.867703       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-891a4d3f-5a1d-4acc-8bc1-19dd56ef9379) returned with <nil>
I1002 14:54:13.867722       1 utils.go:102] GRPC response: {}
I1002 14:54:14.134518       1 controllerserver.go:961] delete snapshot(snapshot-b2f7296f-0ed1-44cd-964a-d071a60b3e9e) under rg(kubetest-xak8dfvt) successfully
I1002 14:54:14.134558       1 utils.go:102] GRPC response: {}
I1002 14:54:18.010195       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:54:18.010222       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19","parameters":{"csi.storage.k8s.io/pv/name":"pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19","csi.storage.k8s.io/pvc/name":"test.csi.azure.com68frm","csi.storage.k8s.io/pvc/namespace":"volume-9045"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:54:18.030283       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:54:18.030367       1 controllerserver.go:271] begin to create azure disk(pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:54:18.030394       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:54:20.381784       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:54:20.381856       1 controllerserver.go:348] create azure disk(pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19 kubernetes.io-created-for-pvc-name:test.csi.azure.com68frm kubernetes.io-created-for-pvc-namespace:volume-9045]) successfully
I1002 14:54:20.381879       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19","csi.storage.k8s.io/pvc/name":"test.csi.azure.com68frm","csi.storage.k8s.io/pvc/namespace":"volume-9045","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19"}}
I1002 14:54:22.722241       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 2 lines ...
I1002 14:54:22.753609       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:54:22.753635       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19" to node "k8s-agentpool1-13787793-vmss000001"
I1002 14:54:22.753654       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:54:22.753667       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19" lun 2 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19  true false 2})]
I1002 14:54:22.753684       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:54:22.753702       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19  true false 2})])
I1002 14:54:22.905943       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:54:32.988570       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:54:32.988616       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 14:54:32.988638       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"2"}}
I1002 14:54:41.290644       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1002 14:54:41.290670       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e3be55d9-4582-4bea-8612-93bc36c67b45"}
I1002 14:54:41.290785       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e3be55d9-4582-4bea-8612-93bc36c67b45 from node k8s-agentpool1-13787793-vmss000001
... skipping 57 lines ...
I1002 14:55:22.385881       1 controllerserver.go:533] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e3be55d9-4582-4bea-8612-93bc36c67b45 from node k8s-agentpool1-13787793-vmss000001 successfully
I1002 14:55:22.385928       1 utils.go:102] GRPC response: {}
I1002 14:55:22.386098       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:55:22.386151       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19" lun 0 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19  true false 0})]
I1002 14:55:22.386213       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:55:22.386227       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19  true false 0})])
I1002 14:55:22.529550       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:55:23.753640       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1002 14:55:23.753667       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-3f580930-36fc-4953-a1c9-e101f467a95f"}
I1002 14:55:23.753786       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-3f580930-36fc-4953-a1c9-e101f467a95f)
I1002 14:55:24.773385       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:55:24.773414       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f","parameters":{"csi.storage.k8s.io/pv/name":"pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f","csi.storage.k8s.io/pvc/name":"test.csi.azure.compssbf","csi.storage.k8s.io/pvc/namespace":"provisioning-1687"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:55:24.795848       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:55:24.795925       1 controllerserver.go:271] begin to create azure disk(pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:55:24.795948       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f StorageAccountType:StandardSSD_LRS Size:5
I1002 14:55:27.088625       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f StorageAccountType:StandardSSD_LRS Size:5
I1002 14:55:27.088689       1 controllerserver.go:348] create azure disk(pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f kubernetes.io-created-for-pvc-name:test.csi.azure.compssbf kubernetes.io-created-for-pvc-namespace:provisioning-1687]) successfully
I1002 14:55:27.088734       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f","csi.storage.k8s.io/pvc/name":"test.csi.azure.compssbf","csi.storage.k8s.io/pvc/namespace":"provisioning-1687","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f"}}
I1002 14:55:28.962058       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-3f580930-36fc-4953-a1c9-e101f467a95f
... skipping 6 lines ...
I1002 14:55:29.553746       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:55:29.623229       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f" to node "k8s-agentpool1-13787793-vmss000001"
I1002 14:55:29.623272       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:55:29.623292       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f" lun 1 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f  true false 1})]
I1002 14:55:29.623309       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:55:29.623323       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f  true false 1})])
I1002 14:55:29.783472       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:55:44.909255       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:55:44.909300       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 14:55:44.909320       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
I1002 14:55:47.482176       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:55:47.482176       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:55:47.482216       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9fa24158-b01a-4725-841c-e55266a60e43","csi.storage.k8s.io/pvc/name":"pvc-m2qqs","csi.storage.k8s.io/pvc/namespace":"volumelimits-4125","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43"}
... skipping 5 lines ...
I1002 14:55:47.508959       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]
I1002 14:55:47.508987       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:55:47.509058       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})])
I1002 14:55:47.580917       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000".
I1002 14:55:47.580961       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:55:47.581006       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:55:47.642835       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:55:47.642874       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:55:47.642946       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:55:47.643008       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:55:47.643060       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:55:47.643110       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:55:47.643187       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 3 lines ...
I1002 14:55:47.723652       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})])
I1002 14:55:47.780951       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:55:47.780991       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 14:55:47.781011       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1002 14:55:47.787791       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:55:47.787815       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19","csi.storage.k8s.io/pvc/name":"test.csi.azure.com68frm","csi.storage.k8s.io/pvc/namespace":"volume-9045","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19"}
I1002 14:55:47.798978       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:55:47.799017       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:55:47.799071       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:55:47.799102       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:55:47.799145       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:55:47.799179       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:55:47.819057       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a39ce3bd-edd5-4e7d-8c71-0b71dc07ff19" to node "k8s-agentpool1-13787793-vmss000001".
... skipping 38 lines ...
I1002 14:56:44.329827       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f)
I1002 14:56:49.514999       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f
I1002 14:56:49.515028       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-b15ce6b2-f45a-4bf1-8fbc-1eeff7a0fb4f) returned with <nil>
I1002 14:56:49.515061       1 utils.go:102] GRPC response: {}
I1002 14:56:55.819909       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:56:55.819936       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4","parameters":{"csi.storage.k8s.io/pv/name":"pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4","csi.storage.k8s.io/pvc/name":"pvc-wqd7c","csi.storage.k8s.io/pvc/namespace":"provisioning-4849"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:56:55.840685       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:56:55.840759       1 controllerserver.go:271] begin to create azure disk(pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:56:55.840785       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:56:58.151500       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4 StorageAccountType:StandardSSD_LRS Size:5
I1002 14:56:58.151546       1 controllerserver.go:348] create azure disk(pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4 kubernetes.io-created-for-pvc-name:pvc-wqd7c kubernetes.io-created-for-pvc-namespace:provisioning-4849]) successfully
I1002 14:56:58.151570       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4","csi.storage.k8s.io/pvc/name":"pvc-wqd7c","csi.storage.k8s.io/pvc/namespace":"provisioning-4849","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4"}}
I1002 14:57:00.537726       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1002 14:57:00.563274       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:57:00.643086       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4" to node "k8s-agentpool1-13787793-vmss000001"
I1002 14:57:00.643131       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:57:00.643153       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4" lun 0 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4  true false 0})]
I1002 14:57:00.643170       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:57:00.643184       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4  true false 0})])
I1002 14:57:00.778968       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:57:15.903675       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:57:15.903720       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 14:57:15.903740       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1002 14:57:31.023687       1 utils.go:95] GRPC call: /csi.v1.Identity/GetPluginInfo
I1002 14:57:31.023717       1 utils.go:96] GRPC request: {}
I1002 14:57:31.023761       1 utils.go:102] GRPC response: {"name":"test.csi.azure.com","vendor_version":"v1.8.0-b1f99a5a1f102374a7a848cd09676c7bf2753657"}
... skipping 15 lines ...
I1002 14:57:32.515109       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:57:32.515155       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:57:32.515172       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:57:32.515186       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]
I1002 14:57:32.515205       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:57:32.515238       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})])
I1002 14:57:32.749206       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:57:32.749240       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:57:32.749295       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:57:32.749325       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:57:32.749361       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:57:32.749397       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:57:33.305666       1 controllerserver.go:918] create snapshot(snapshot-03f0ba31-7c0a-4ad7-b0db-f269819facef) under rg(kubetest-xak8dfvt) successfully
I1002 14:57:33.326037       1 utils.go:102] GRPC response: {"snapshot":{"creation_time":{"nanos":198031900,"seconds":1633186651},"ready_to_use":true,"size_bytes":5368709120,"snapshot_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/snapshots/snapshot-03f0ba31-7c0a-4ad7-b0db-f269819facef","source_volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4"}}
I1002 14:57:35.759837       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:57:35.759866       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d","csi.storage.k8s.io/pvc/name":"pvc-667l4","csi.storage.k8s.io/pvc/namespace":"provisioning-4849"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}],"volume_content_source":{"Type":{"Snapshot":{"snapshot_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/snapshots/snapshot-03f0ba31-7c0a-4ad7-b0db-f269819facef"}}}}
I1002 14:57:35.778963       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:57:35.779051       1 controllerserver.go:271] begin to create azure disk(pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:57:35.779085       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d StorageAccountType:StandardSSD_LRS Size:5
I1002 14:57:38.148281       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d StorageAccountType:StandardSSD_LRS Size:5
I1002 14:57:38.148339       1 controllerserver.go:348] create azure disk(pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d kubernetes.io-created-for-pvc-name:pvc-667l4 kubernetes.io-created-for-pvc-namespace:provisioning-4849]) successfully
I1002 14:57:38.148386       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":{"Snapshot":{"snapshot_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/snapshots/snapshot-03f0ba31-7c0a-4ad7-b0db-f269819facef"}}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d","csi.storage.k8s.io/pvc/name":"pvc-667l4","csi.storage.k8s.io/pvc/namespace":"provisioning-4849","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d"}}
I1002 14:57:40.203939       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1002 14:57:40.274938       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 14:57:40.353595       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000"
I1002 14:57:40.353656       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:57:40.353706       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]
I1002 14:57:40.353740       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:57:40.353774       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})])
I1002 14:57:40.451598       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:57:40.451635       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 14:57:40.451694       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 14:57:40.451738       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 14:57:40.451776       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 14:57:40.451810       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 14:57:40.525358       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 12 lines ...
I1002 14:57:47.704547       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4 from node k8s-agentpool1-13787793-vmss000001
I1002 14:57:47.704573       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:57:47.772647       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d" lun 0 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d  true false 0})]
I1002 14:57:47.772690       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:57:47.772713       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d  true false 0})])
I1002 14:57:47.772735       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:57:48.160232       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:57:48.160440       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4" from node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4:pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4]
I1002 14:57:48.160486       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
E1002 14:57:48.217167       1 azure_controller_vmss.go:183] detach azure disk on node(k8s-agentpool1-13787793-vmss000001): disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4:pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4]) not found
I1002 14:57:48.217209       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4:pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4])
I1002 14:57:58.290658       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:57:58.290724       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ecee0bfb-93c6-493e-a315-cb4d387f323d" to node "k8s-agentpool1-13787793-vmss000001" successfully
... skipping 51 lines ...
I1002 14:59:12.441025       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-eb1b36de-7421-4fcb-877b-401a0877f3a4) returned with <nil>
I1002 14:59:12.441045       1 utils.go:102] GRPC response: {}
I1002 14:59:12.751292       1 controllerserver.go:961] delete snapshot(snapshot-03f0ba31-7c0a-4ad7-b0db-f269819facef) under rg(kubetest-xak8dfvt) successfully
I1002 14:59:12.751328       1 utils.go:102] GRPC response: {}
I1002 14:59:15.618310       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 14:59:15.618337       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd","parameters":{"csi.storage.k8s.io/pv/name":"pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd","csi.storage.k8s.io/pvc/name":"test.csi.azure.combtkt7","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-8187"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 14:59:15.641503       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 14:59:15.641623       1 controllerserver.go:271] begin to create azure disk(pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 14:59:15.641641       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd StorageAccountType:StandardSSD_LRS Size:5
I1002 14:59:17.989425       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd StorageAccountType:StandardSSD_LRS Size:5
I1002 14:59:17.989469       1 controllerserver.go:348] create azure disk(pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd kubernetes.io-created-for-pvc-name:test.csi.azure.combtkt7 kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-8187]) successfully
I1002 14:59:17.989495       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd","csi.storage.k8s.io/pvc/name":"test.csi.azure.combtkt7","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-8187","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd"}}
I1002 14:59:20.367512       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1002 14:59:20.400854       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 14:59:20.456010       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd" to node "k8s-agentpool1-13787793-vmss000001"
I1002 14:59:20.456047       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:59:20.456066       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd" lun 0 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd  true false 0})]
I1002 14:59:20.456082       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 14:59:20.456094       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd  true false 0})])
I1002 14:59:20.610640       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 14:59:30.692452       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 14:59:30.692495       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 14:59:30.692514       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1002 14:59:30.707848       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 14:59:30.707869       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd","csi.storage.k8s.io/pvc/name":"test.csi.azure.combtkt7","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-8187","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd"}
I1002 14:59:30.732362       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd" to node "k8s-agentpool1-13787793-vmss000001".
... skipping 19 lines ...
I1002 15:00:45.242619       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd)
I1002 15:00:50.425469       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd
I1002 15:00:50.425515       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-72e85fd4-7fc2-4d83-8c60-94d53daf4bdd) returned with <nil>
I1002 15:00:50.425550       1 utils.go:102] GRPC response: {}
I1002 15:00:56.631554       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 15:00:56.631582       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b","parameters":{"csi.storage.k8s.io/pv/name":"pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b","csi.storage.k8s.io/pvc/name":"test.csi.azure.comhbl7j","csi.storage.k8s.io/pvc/namespace":"provisioning-2365"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 15:00:56.651790       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 15:00:56.651898       1 controllerserver.go:271] begin to create azure disk(pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 15:00:56.651930       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b StorageAccountType:StandardSSD_LRS Size:5
I1002 15:00:58.985449       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b StorageAccountType:StandardSSD_LRS Size:5
I1002 15:00:58.985521       1 controllerserver.go:348] create azure disk(pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b kubernetes.io-created-for-pvc-name:test.csi.azure.comhbl7j kubernetes.io-created-for-pvc-namespace:provisioning-2365]) successfully
I1002 15:00:58.985545       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b","csi.storage.k8s.io/pvc/name":"test.csi.azure.comhbl7j","csi.storage.k8s.io/pvc/namespace":"provisioning-2365","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b"}}
I1002 15:01:01.307881       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1002 15:01:01.335324       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 15:01:01.450318       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b" to node "k8s-agentpool1-13787793-vmss000001"
I1002 15:01:01.450363       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:01:01.450384       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b" lun 0 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b  true false 0})]
I1002 15:01:01.450403       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:01:01.450433       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b  true false 0})])
I1002 15:01:01.624957       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 15:01:11.728694       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 15:01:11.728752       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 15:01:11.728790       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1002 15:01:33.844246       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1002 15:01:33.844270       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b"}
I1002 15:01:33.844397       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2fce7c93-5ffa-4ea2-b6e7-196afc9f770b from node k8s-agentpool1-13787793-vmss000001
... skipping 32 lines ...
I1002 15:02:32.783038       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:02:32.783076       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to node "k8s-agentpool1-13787793-vmss000000"
I1002 15:02:32.783092       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:02:32.783105       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]
I1002 15:02:32.783137       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:02:32.783168       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})])
I1002 15:02:32.855386       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 15:02:32.855530       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 15:02:32.855589       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 15:02:32.855627       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 15:02:32.855662       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 15:02:32.855686       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 15:02:40.465308       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1002 15:02:40.507810       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 15:02:40.582604       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000"
I1002 15:02:40.582669       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:02:40.582689       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]
I1002 15:02:40.582708       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:02:40.582739       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})])
I1002 15:02:40.646489       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 15:02:40.646541       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 15:02:40.646593       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 15:02:40.646662       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 15:02:40.646702       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 15:02:40.646727       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 15:02:49.683652       1 azure_vmss.go:319] Couldn't find VMSS VM with scaleSetName "k8s-agentpool1-13787793-vmss" and instanceID "0", refreshing the cache
... skipping 14 lines ...
I1002 15:05:14.557861       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3)
I1002 15:05:15.078786       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3
I1002 15:05:15.078816       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-d9af5ce6-67d9-4259-b8b3-7d97e0914be3) returned with <nil>
I1002 15:05:15.078850       1 utils.go:102] GRPC response: {}
I1002 15:05:24.397331       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 15:05:24.397359       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39","csi.storage.k8s.io/pvc/name":"test.csi.azure.com95bkn","csi.storage.k8s.io/pvc/namespace":"provisioning-1422"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 15:05:24.418214       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 15:05:24.418285       1 controllerserver.go:271] begin to create azure disk(pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 15:05:24.418305       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39 StorageAccountType:StandardSSD_LRS Size:5
I1002 15:05:26.763119       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39 StorageAccountType:StandardSSD_LRS Size:5
I1002 15:05:26.763164       1 controllerserver.go:348] create azure disk(pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39 kubernetes.io-created-for-pvc-name:test.csi.azure.com95bkn kubernetes.io-created-for-pvc-namespace:provisioning-1422]) successfully
I1002 15:05:26.763191       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39","csi.storage.k8s.io/pvc/name":"test.csi.azure.com95bkn","csi.storage.k8s.io/pvc/namespace":"provisioning-1422","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39"}}
I1002 15:05:29.136886       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 2 lines ...
I1002 15:05:29.163858       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:05:29.163896       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39" to node "k8s-agentpool1-13787793-vmss000001"
I1002 15:05:29.163923       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:05:29.163939       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39" lun 0 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39  true false 0})]
I1002 15:05:29.163969       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:05:29.163999       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39  true false 0})])
I1002 15:05:29.397923       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 15:05:39.497918       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 15:05:39.497957       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-ab7a0cd3-e9e2-49ff-b2cd-0b8feb86dd39" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 15:05:39.497997       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1002 15:05:47.482862       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 15:05:47.482880       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 15:05:47.482911       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-cc756372-10e6-4c4a-9a17-148d8582402b","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-8ms2w-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-6615","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b"}
... skipping 6 lines ...
I1002 15:05:47.700933       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" to node "k8s-agentpool1-13787793-vmss000000"
I1002 15:05:47.700983       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:05:47.701044       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to node "k8s-agentpool1-13787793-vmss000000"
I1002 15:05:47.701085       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" lun 0 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 0})]
I1002 15:05:47.701202       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:05:47.701396       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 0})])
I1002 15:05:48.519688       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cc756372-10e6-4c4a-9a17-148d8582402b  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 15:05:48.519878       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:05:48.519921       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 15:05:48.573006       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" lun 32 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]
I1002 15:05:48.573065       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:05:48.573124       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})])
I1002 15:05:48.638123       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 15:05:48.638153       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-13787793-vmss/virtualMachines/0, error: %!s(<nil>)
E1002 15:05:48.638210       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})]) on rg(kubetest-xak8dfvt) vm(k8s-agentpool1-13787793-vmss000000) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1002 15:05:48.638241       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1002 15:05:48.638277       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1002 15:05:48.638314       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to instance "k8s-agentpool1-13787793-vmss000000" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1002 15:05:50.064723       1 azure_vmss.go:319] Couldn't find VMSS VM with scaleSetName "k8s-agentpool1-13787793-vmss" and instanceID "0", refreshing the cache
... skipping 62 lines ...
I1002 15:07:46.630587       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b from node k8s-agentpool1-13787793-vmss000000
I1002 15:07:46.630605       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:07:46.727465       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" lun 0 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 0})]
I1002 15:07:46.727523       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:07:46.727593       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:07:46.727596       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 0})])
I1002 15:07:46.912498       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fa24158-b01a-4725-841c-e55266a60e43  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 15:07:46.912675       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b" from node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:pvc-cc756372-10e6-4c4a-9a17-148d8582402b]
I1002 15:07:46.912742       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
E1002 15:07:47.039768       1 azure_controller_vmss.go:183] detach azure disk on node(k8s-agentpool1-13787793-vmss000000): disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:pvc-cc756372-10e6-4c4a-9a17-148d8582402b]) not found
I1002 15:07:47.039803       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cc756372-10e6-4c4a-9a17-148d8582402b:pvc-cc756372-10e6-4c4a-9a17-148d8582402b])
I1002 15:08:07.104568       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" attached to node "k8s-agentpool1-13787793-vmss000000".
I1002 15:08:07.104614       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-9fa24158-b01a-4725-841c-e55266a60e43" to node "k8s-agentpool1-13787793-vmss000000" successfully
... skipping 607 lines ...
I1002 15:13:00.012170       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7112c99e-555c-4139-ba7f-ae80c671397d)
I1002 15:13:05.206555       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7112c99e-555c-4139-ba7f-ae80c671397d
I1002 15:13:05.206594       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7112c99e-555c-4139-ba7f-ae80c671397d) returned with <nil>
I1002 15:13:05.206642       1 utils.go:102] GRPC response: {}
I1002 15:13:12.100866       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 15:13:12.100893       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8","parameters":{"csi.storage.k8s.io/pv/name":"pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8","csi.storage.k8s.io/pvc/name":"inline-volume-tester-rkjl8-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2716"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 15:13:12.121188       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 15:13:12.121250       1 controllerserver.go:271] begin to create azure disk(pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) diskZone() maxShares(0)
I1002 15:13:12.121265       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8 StorageAccountType:StandardSSD_LRS Size:5
I1002 15:13:14.456864       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8 StorageAccountType:StandardSSD_LRS Size:5
I1002 15:13:14.456909       1 controllerserver.go:348] create azure disk(pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8 kubernetes.io-created-for-pvc-name:inline-volume-tester-rkjl8-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-2716]) successfully
I1002 15:13:14.456937       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8","csi.storage.k8s.io/pvc/name":"inline-volume-tester-rkjl8-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2716","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8"}}
I1002 15:13:15.224757       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1002 15:13:15.248123       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 15:13:15.320924       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8" to node "k8s-agentpool1-13787793-vmss000000"
I1002 15:13:15.320976       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:13:15.321000       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8" lun 0 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8  true false 0})]
I1002 15:13:15.321017       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:13:15.321031       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8  true false 0})])
I1002 15:13:15.652748       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 15:13:25.790581       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8" attached to node "k8s-agentpool1-13787793-vmss000000".
I1002 15:13:25.790672       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8" to node "k8s-agentpool1-13787793-vmss000000" successfully
I1002 15:13:25.790693       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1002 15:13:25.799044       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 15:13:25.799065       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8","csi.storage.k8s.io/pvc/name":"inline-volume-tester-rkjl8-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2716","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8"}
I1002 15:13:25.858566       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8" to node "k8s-agentpool1-13787793-vmss000000".
... skipping 19 lines ...
I1002 15:14:39.631221       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8)
I1002 15:14:44.855143       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8
I1002 15:14:44.855176       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-722db8b4-450c-4bc5-ac37-7542182b9bf8) returned with <nil>
I1002 15:14:44.855211       1 utils.go:102] GRPC response: {}
I1002 15:14:49.444546       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 15:14:49.444576       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-7c0e9740-944b-4c01-ba89-823af906d09a","parameters":{"csi.storage.k8s.io/pv/name":"pvc-7c0e9740-944b-4c01-ba89-823af906d09a","csi.storage.k8s.io/pvc/name":"pvc-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 15:14:49.464261       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7c0e9740-944b-4c01-ba89-823af906d09a, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-7c0e9740-944b-4c01-ba89-823af906d09a' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 15:14:49.464366       1 controllerserver.go:271] begin to create azure disk(pvc-7c0e9740-944b-4c01-ba89-823af906d09a) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(10) diskZone() maxShares(0)
I1002 15:14:49.464381       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-7c0e9740-944b-4c01-ba89-823af906d09a StorageAccountType:StandardSSD_LRS Size:10
I1002 15:14:51.793955       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-7c0e9740-944b-4c01-ba89-823af906d09a StorageAccountType:StandardSSD_LRS Size:10
I1002 15:14:51.794022       1 controllerserver.go:348] create azure disk(pvc-7c0e9740-944b-4c01-ba89-823af906d09a) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-7c0e9740-944b-4c01-ba89-823af906d09a kubernetes.io-created-for-pvc-name:pvc-azuredisk kubernetes.io-created-for-pvc-namespace:default]) successfully
I1002 15:14:51.794061       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7c0e9740-944b-4c01-ba89-823af906d09a","csi.storage.k8s.io/pvc/name":"pvc-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7c0e9740-944b-4c01-ba89-823af906d09a"}}
I1002 15:14:54.306143       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1002 15:14:54.329693       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 15:14:54.410507       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7c0e9740-944b-4c01-ba89-823af906d09a" to node "k8s-agentpool1-13787793-vmss000000"
I1002 15:14:54.410575       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:14:54.410598       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7c0e9740-944b-4c01-ba89-823af906d09a" lun 0 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7c0e9740-944b-4c01-ba89-823af906d09a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7c0e9740-944b-4c01-ba89-823af906d09a  true false 0})]
I1002 15:14:54.410618       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:14:54.410631       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7c0e9740-944b-4c01-ba89-823af906d09a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7c0e9740-944b-4c01-ba89-823af906d09a  true false 0})])
I1002 15:14:54.560276       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-7c0e9740-944b-4c01-ba89-823af906d09a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7c0e9740-944b-4c01-ba89-823af906d09a  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 15:15:04.644499       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7c0e9740-944b-4c01-ba89-823af906d09a" attached to node "k8s-agentpool1-13787793-vmss000000".
I1002 15:15:04.644535       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-7c0e9740-944b-4c01-ba89-823af906d09a" to node "k8s-agentpool1-13787793-vmss000000" successfully
I1002 15:15:04.644554       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1002 15:15:16.057735       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 15:15:16.057763       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7","parameters":{"csi.storage.k8s.io/pv/name":"pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-0","csi.storage.k8s.io/pvc/namespace":"default","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 15:15:16.077274       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 15:15:16.077443       1 controllerserver.go:271] begin to create azure disk(pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(10) diskZone() maxShares(0)
I1002 15:15:16.077481       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7 StorageAccountType:StandardSSD_LRS Size:10
I1002 15:15:18.430727       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7 StorageAccountType:StandardSSD_LRS Size:10
I1002 15:15:18.430805       1 controllerserver.go:348] create azure disk(pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7 kubernetes.io-created-for-pvc-name:persistent-storage-statefulset-azuredisk-0 kubernetes.io-created-for-pvc-namespace:default]) successfully
I1002 15:15:18.430832       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7"}}
I1002 15:15:19.280011       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 2 lines ...
I1002 15:15:19.437853       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:15:19.437916       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7" to node "k8s-agentpool1-13787793-vmss000001"
I1002 15:15:19.437974       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:15:19.437986       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7" lun 0 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7  true false 0})]
I1002 15:15:19.438060       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:15:19.438076       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7  true false 0})])
I1002 15:15:19.685379       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 15:15:34.865419       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 15:15:34.865465       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 15:15:34.865485       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1002 15:15:34.874090       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 15:15:34.874112       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7"}
I1002 15:15:34.895706       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7" to node "k8s-agentpool1-13787793-vmss000001".
I1002 15:15:34.895750       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:15:34.895772       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000001, refreshing the cache
I1002 15:15:34.969992       1 azure_controller_common.go:457] azureDisk - find disk: lun 0 name "pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7"
I1002 15:15:34.970048       1 controllerserver.go:455] Attach operation is successful. volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-a3aaa3ce-e9d8-4437-82f7-aa4c0384b1a7" is already attached to node "k8s-agentpool1-13787793-vmss000001" at lun 0.
I1002 15:15:34.970070       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1002 15:16:00.693073       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 15:16:00.693105       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-8318813a-4038-419b-bc16-0399eb6c1c40","parameters":{"csi.storage.k8s.io/pv/name":"pvc-8318813a-4038-419b-bc16-0399eb6c1c40","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-nonroot-0","csi.storage.k8s.io/pvc/namespace":"default","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 15:16:00.715142       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8318813a-4038-419b-bc16-0399eb6c1c40, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-8318813a-4038-419b-bc16-0399eb6c1c40' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 15:16:00.715262       1 controllerserver.go:271] begin to create azure disk(pvc-8318813a-4038-419b-bc16-0399eb6c1c40) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(10) diskZone() maxShares(0)
I1002 15:16:00.715277       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-8318813a-4038-419b-bc16-0399eb6c1c40 StorageAccountType:StandardSSD_LRS Size:10
I1002 15:16:03.077002       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-8318813a-4038-419b-bc16-0399eb6c1c40 StorageAccountType:StandardSSD_LRS Size:10
I1002 15:16:03.077063       1 controllerserver.go:348] create azure disk(pvc-8318813a-4038-419b-bc16-0399eb6c1c40) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-8318813a-4038-419b-bc16-0399eb6c1c40 kubernetes.io-created-for-pvc-name:persistent-storage-statefulset-azuredisk-nonroot-0 kubernetes.io-created-for-pvc-namespace:default]) successfully
I1002 15:16:03.077089       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-8318813a-4038-419b-bc16-0399eb6c1c40","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-nonroot-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8318813a-4038-419b-bc16-0399eb6c1c40"}}
I1002 15:16:05.318995       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 2 lines ...
I1002 15:16:05.344444       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:16:05.344497       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8318813a-4038-419b-bc16-0399eb6c1c40" to node "k8s-agentpool1-13787793-vmss000000"
I1002 15:16:05.344514       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:16:05.344545       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8318813a-4038-419b-bc16-0399eb6c1c40" lun 1 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8318813a-4038-419b-bc16-0399eb6c1c40:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8318813a-4038-419b-bc16-0399eb6c1c40  true false 1})]
I1002 15:16:05.344603       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:16:05.344620       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8318813a-4038-419b-bc16-0399eb6c1c40:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8318813a-4038-419b-bc16-0399eb6c1c40  true false 1})])
I1002 15:16:05.509665       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8318813a-4038-419b-bc16-0399eb6c1c40:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8318813a-4038-419b-bc16-0399eb6c1c40  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 15:16:20.691079       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8318813a-4038-419b-bc16-0399eb6c1c40" attached to node "k8s-agentpool1-13787793-vmss000000".
I1002 15:16:20.691121       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8318813a-4038-419b-bc16-0399eb6c1c40" to node "k8s-agentpool1-13787793-vmss000000" successfully
I1002 15:16:20.691159       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
I1002 15:16:20.700305       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 15:16:20.700330       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-8318813a-4038-419b-bc16-0399eb6c1c40","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-nonroot-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8318813a-4038-419b-bc16-0399eb6c1c40"}
I1002 15:16:20.724486       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8318813a-4038-419b-bc16-0399eb6c1c40" to node "k8s-agentpool1-13787793-vmss000000".
I1002 15:16:20.724518       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:16:20.724540       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-13787793-vmss000000, refreshing the cache
I1002 15:16:20.809797       1 azure_controller_common.go:457] azureDisk - find disk: lun 1 name "pvc-8318813a-4038-419b-bc16-0399eb6c1c40" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8318813a-4038-419b-bc16-0399eb6c1c40"
I1002 15:16:20.809849       1 controllerserver.go:455] Attach operation is successful. volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8318813a-4038-419b-bc16-0399eb6c1c40" is already attached to node "k8s-agentpool1-13787793-vmss000000" at lun 1.
I1002 15:16:20.809874       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
I1002 15:16:29.936441       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 15:16:29.936468       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54","parameters":{"csi.storage.k8s.io/pv/name":"pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54","csi.storage.k8s.io/pvc/name":"nginx-azuredisk-ephemeral-azuredisk01","csi.storage.k8s.io/pvc/namespace":"default","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 15:16:29.958641       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 15:16:29.958708       1 controllerserver.go:271] begin to create azure disk(pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(10) diskZone() maxShares(0)
I1002 15:16:29.958732       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54 StorageAccountType:StandardSSD_LRS Size:10
I1002 15:16:32.341562       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54 StorageAccountType:StandardSSD_LRS Size:10
I1002 15:16:32.341626       1 controllerserver.go:348] create azure disk(pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54 kubernetes.io-created-for-pvc-name:nginx-azuredisk-ephemeral-azuredisk01 kubernetes.io-created-for-pvc-namespace:default]) successfully
I1002 15:16:32.341693       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54","csi.storage.k8s.io/pvc/name":"nginx-azuredisk-ephemeral-azuredisk01","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54"}}
I1002 15:16:33.338238       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 2 lines ...
I1002 15:16:33.363622       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:16:33.363661       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54" to node "k8s-agentpool1-13787793-vmss000001"
I1002 15:16:33.363705       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:16:33.363716       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54" lun 1 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54  true false 1})]
I1002 15:16:33.363773       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:16:33.363804       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54  true false 1})])
I1002 15:16:33.531757       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 15:16:48.712188       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 15:16:48.712232       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-18b0c08f-b70d-445f-b9f3-d8ff12c7bc54" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 15:16:48.712252       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
I1002 15:16:51.167335       1 azure_vmss.go:319] Couldn't find VMSS VM with scaleSetName "k8s-agentpool1-13787793-vmss" and instanceID "1", refreshing the cache
I1002 15:16:54.329750       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 15:16:54.329808       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-cf268a0f-7638-47df-b926-8d6780f78065","parameters":{"csi.storage.k8s.io/pv/name":"pvc-cf268a0f-7638-47df-b926-8d6780f78065","csi.storage.k8s.io/pvc/name":"daemonset-azuredisk-ephemeral-8ntdd-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 15:16:54.333713       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1002 15:16:54.333734       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3","parameters":{"csi.storage.k8s.io/pv/name":"pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3","csi.storage.k8s.io/pvc/name":"daemonset-azuredisk-ephemeral-pmnbr-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1002 15:16:54.349484       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cf268a0f-7638-47df-b926-8d6780f78065, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-cf268a0f-7638-47df-b926-8d6780f78065' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 15:16:54.349543       1 controllerserver.go:271] begin to create azure disk(pvc-cf268a0f-7638-47df-b926-8d6780f78065) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(10) diskZone() maxShares(0)
I1002 15:16:54.349563       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-cf268a0f-7638-47df-b926-8d6780f78065 StorageAccountType:StandardSSD_LRS Size:10
I1002 15:16:54.380232       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3' under resource group 'kubetest-xak8dfvt' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1002 15:16:54.380290       1 controllerserver.go:271] begin to create azure disk(pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(10) diskZone() maxShares(0)
I1002 15:16:54.380323       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3 StorageAccountType:StandardSSD_LRS Size:10
I1002 15:16:56.728150       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-cf268a0f-7638-47df-b926-8d6780f78065 StorageAccountType:StandardSSD_LRS Size:10
I1002 15:16:56.728248       1 controllerserver.go:348] create azure disk(pvc-cf268a0f-7638-47df-b926-8d6780f78065) account type(StandardSSD_LRS) rg(kubetest-xak8dfvt) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-cf268a0f-7638-47df-b926-8d6780f78065 kubernetes.io-created-for-pvc-name:daemonset-azuredisk-ephemeral-8ntdd-azuredisk kubernetes.io-created-for-pvc-namespace:default]) successfully
I1002 15:16:56.728288       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-cf268a0f-7638-47df-b926-8d6780f78065","csi.storage.k8s.io/pvc/name":"daemonset-azuredisk-ephemeral-8ntdd-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cf268a0f-7638-47df-b926-8d6780f78065"}}
I1002 15:16:56.736925       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3 StorageAccountType:StandardSSD_LRS Size:10
... skipping 5 lines ...
I1002 15:16:58.429991       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:16:58.430070       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cf268a0f-7638-47df-b926-8d6780f78065" to node "k8s-agentpool1-13787793-vmss000000"
I1002 15:16:58.430146       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:16:58.430165       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-cf268a0f-7638-47df-b926-8d6780f78065" lun 2 to node "k8s-agentpool1-13787793-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cf268a0f-7638-47df-b926-8d6780f78065:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cf268a0f-7638-47df-b926-8d6780f78065  true false 2})]
I1002 15:16:58.430246       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:16:58.430279       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cf268a0f-7638-47df-b926-8d6780f78065:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cf268a0f-7638-47df-b926-8d6780f78065  true false 2})])
I1002 15:16:58.581751       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-cf268a0f-7638-47df-b926-8d6780f78065:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cf268a0f-7638-47df-b926-8d6780f78065  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 15:16:59.357416       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 15:16:59.357465       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3","csi.storage.k8s.io/pvc/name":"daemonset-azuredisk-ephemeral-pmnbr-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3"}
I1002 15:16:59.380440       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3" to node "k8s-agentpool1-13787793-vmss000001".
I1002 15:16:59.380474       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:16:59.380499       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3" to node "k8s-agentpool1-13787793-vmss000001"
I1002 15:16:59.380516       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:16:59.380528       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3" lun 2 to node "k8s-agentpool1-13787793-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3  true false 2})]
I1002 15:16:59.380545       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-13787793-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1002 15:16:59.380561       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3  true false 2})])
I1002 15:16:59.543441       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-xak8dfvt): vm(k8s-agentpool1-13787793-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-xak8dfvt/providers/microsoft.compute/disks/pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1002 15:17:09.621709       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3" attached to node "k8s-agentpool1-13787793-vmss000001".
I1002 15:17:09.621755       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3" to node "k8s-agentpool1-13787793-vmss000001" successfully
I1002 15:17:09.621776       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"2"}}
I1002 15:17:09.629836       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1002 15:17:09.629866       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-13787793-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3","csi.storage.k8s.io/pvc/name":"daemonset-azuredisk-ephemeral-pmnbr-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1633185947869-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3"}
I1002 15:17:09.667149       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-8cf81b79-1925-4563-89af-5dd3a6ea66c3" to node "k8s-agentpool1-13787793-vmss000001".
... skipping 27 lines ...
Platform: linux/amd64
Topology Key: topology.test.csi.azure.com/zone

Streaming logs below:
I1002 14:46:06.885957       1 azuredisk.go:139] driver userAgent: test.csi.azure.com/v1.8.0-b1f99a5a1f102374a7a848cd09676c7bf2753657 gc/go1.17.1 (amd64-linux) e2e-test
I1002 14:46:06.887149       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W1002 14:46:07.008357       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I1002 14:46:07.008383       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I1002 14:46:07.008389       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I1002 14:46:07.008420       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I1002 14:46:07.008980       1 azure_auth.go:234] Using AzurePublicCloud environment
I1002 14:46:07.008991       1 azure_auth.go:96] azure: using managed identity extension to retrieve access token
I1002 14:46:07.008995       1 azure_auth.go:102] azure: using User Assigned MSI ID to retrieve access token
... skipping 102 lines ...
Platform: linux/amd64
Topology Key: topology.test.csi.azure.com/zone

Streaming logs below:
I1002 14:46:01.386724       1 azuredisk.go:139] driver userAgent: test.csi.azure.com/v1.8.0-b1f99a5a1f102374a7a848cd09676c7bf2753657 gc/go1.17.1 (amd64-linux) e2e-test
I1002 14:46:01.387843       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W1002 14:46:01.496961       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I1002 14:46:01.496991       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I1002 14:46:01.497018       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I1002 14:46:01.497060       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I1002 14:46:01.497941       1 azure_auth.go:234] Using AzurePublicCloud environment
I1002 14:46:01.497964       1 azure_auth.go:96] azure: using managed identity extension to retrieve access token
I1002 14:46:01.497969       1 azure_auth.go:102] azure: using User Assigned MSI ID to retrieve access token
... skipping 513 lines ...
I1002 14:49:37.226051       1 azure_common_linux.go:177] azureDisk - found /dev/disk/azure/scsi1/lun23 by sdae under /dev/disk/azure/scsi1/
I1002 14:49:37.226183       1 nodeserver.go:111] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun23. perfProfile none accountType 
I1002 14:49:37.226225       1 nodeserver.go:152] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun23 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ad89e320-0715-45d1-a22c-177c4c9109c3/globalmount with mount options([])
I1002 14:49:37.226237       1 mount_linux.go:446] Attempting to determine if disk "/dev/disk/azure/scsi1/lun23" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun23])
I1002 14:49:37.237513       1 mount_linux.go:449] Output: ""
I1002 14:49:37.237553       1 mount_linux.go:408] Disk "/dev/disk/azure/scsi1/lun23" appears to be unformatted, attempting to format as type: "ext4" with options: [-F -m0 /dev/disk/azure/scsi1/lun23]
W1002 14:49:37.447514       1 azure_common_linux.go:180] azureDisk - getDiskLinkByDevName by sdag under /dev/disk/azure/scsi1/ failed, error: device name(sdag) is not found under /dev/disk/azure/scsi1/
W1002 14:49:37.448032       1 azure_common_linux.go:180] azureDisk - getDiskLinkByDevName by sdag under /dev/disk/by-id/ failed, error: device name(sdag) is not found under /dev/disk/by-id/
I1002 14:49:37.448134       1 nodeserver.go:111] NodeStageVolume: perf optimization is disabled for /dev/sdag. perfProfile none accountType 
I1002 14:49:37.448194       1 nodeserver.go:152] NodeStageVolume: formatting /dev/sdag and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-2a16a0b0-b771-46cf-a4e2-68f44b291606/globalmount with mount options([])
I1002 14:49:37.448221       1 mount_linux.go:446] Attempting to determine if disk "/dev/sdag" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/sdag])
I1002 14:49:37.455029       1 mount_linux.go:449] Output: ""
I1002 14:49:37.455053       1 mount_linux.go:408] Disk "/dev/sdag" appears to be unformatted, attempting to format as type: "ext4" with options: [-F -m0 /dev/sdag]
I1002 14:49:37.467960       1 azure_common_linux.go:177] azureDisk - found /dev/disk/azure/scsi1/lun21 by sdaf under /dev/disk/azure/scsi1/
... skipping 598 lines ...
I1002 15:11:31.281093       1 nodeserver.go:195] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40/globalmount
I1002 15:11:31.281125       1 mount_helper_common.go:99] "/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40/globalmount" is a mountpoint, unmounting
I1002 15:11:31.281133       1 mount_linux.go:266] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-2dea6350-9b3b-4a47-8729-660b4a91ab40/globalmount
W1002 15:11:31.281512       1 mount_helper_common.go:129] Warning: "/var/lib/kubelet/pods/00e6694c-dd52-4882-a0df-ba36fceada44/volumes/kubernetes.io~csi/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b/mount" is not a mountpoint, deleting
I1002 15:11:31.281572       1 nodeserver.go:290] NodeUnpublishVolume: unmount volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b on /var/lib/kubelet/pods/00e6694c-dd52-4882-a0df-ba36fceada44/volumes/kubernetes.io~csi/pvc-e19708a1-7049-4d60-a087-03a7611a7f3b/mount successfully
I1002 15:11:31.281588       1 utils.go:102] GRPC response: {}
E1002 15:11:31.281675       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to unmount target "/var/lib/kubelet/pods/00e6694c-dd52-4882-a0df-ba36fceada44/volumes/kubernetes.io~csi/pvc-4d955fb5-be00-47a9-a0a9-cc1f84edbd17/mount": could not get consistent content of /proc/mounts after 3 attempts
W1002 15:11:31.281717       1 mount_helper_common.go:129] Warning: "/var/lib/kubelet/pods/00e6694c-dd52-4882-a0df-ba36fceada44/volumes/kubernetes.io~csi/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d/mount" is not a mountpoint, deleting
I1002 15:11:31.281772       1 nodeserver.go:290] NodeUnpublishVolume: unmount volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d on /var/lib/kubelet/pods/00e6694c-dd52-4882-a0df-ba36fceada44/volumes/kubernetes.io~csi/pvc-af445f76-0f4c-4b5e-8baa-e6d284f0777d/mount successfully
I1002 15:11:31.281783       1 utils.go:102] GRPC response: {}
W1002 15:11:31.282137       1 mount_helper_common.go:129] Warning: "/var/lib/kubelet/pods/00e6694c-dd52-4882-a0df-ba36fceada44/volumes/kubernetes.io~csi/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e/mount" is not a mountpoint, deleting
I1002 15:11:31.282221       1 nodeserver.go:290] NodeUnpublishVolume: unmount volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-xak8dfvt/providers/Microsoft.Compute/disks/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e on /var/lib/kubelet/pods/00e6694c-dd52-4882-a0df-ba36fceada44/volumes/kubernetes.io~csi/pvc-2ada027c-fb64-42ef-9797-f62647ebd42e/mount successfully
I1002 15:11:31.282238       1 utils.go:102] GRPC response: {}
... skipping 403 lines ...
Platform: linux/amd64
Topology Key: topology.test.csi.azure.com/zone

Streaming logs below:
I1002 14:45:37.209700       1 azuredisk.go:139] driver userAgent: test.csi.azure.com/v1.8.0-b1f99a5a1f102374a7a848cd09676c7bf2753657 gc/go1.17.1 (amd64-linux) e2e-test
I1002 14:45:37.211490       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W1002 14:45:37.237607       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I1002 14:45:37.237641       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I1002 14:45:37.237649       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I1002 14:45:37.237679       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I1002 14:45:37.239046       1 azure_auth.go:234] Using AzurePublicCloud environment
I1002 14:45:37.239070       1 azure_auth.go:96] azure: using managed identity extension to retrieve access token
I1002 14:45:37.239075       1 azure_auth.go:102] azure: using User Assigned MSI ID to retrieve access token
... skipping 72 lines ...
Platform: linux/amd64
Topology Key: topology.test.csi.azure.com/zone

Streaming logs below:
I1002 14:45:41.288917       1 azuredisk.go:139] driver userAgent: test.csi.azure.com/v1.8.0-b1f99a5a1f102374a7a848cd09676c7bf2753657 gc/go1.17.1 (amd64-linux) e2e-test
I1002 14:45:41.290571       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W1002 14:45:41.387266       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I1002 14:45:41.387295       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I1002 14:45:41.387304       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I1002 14:45:41.387428       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I1002 14:45:41.388387       1 azure_auth.go:234] Using AzurePublicCloud environment
I1002 14:45:41.388412       1 azure_auth.go:96] azure: using managed identity extension to retrieve access token
I1002 14:45:41.388418       1 azure_auth.go:102] azure: using User Assigned MSI ID to retrieve access token
... skipping 1533 lines ...
cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-xak8dfvt",source="test.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="300"} 80
cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-xak8dfvt",source="test.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="600"} 80
cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-xak8dfvt",source="test.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="1200"} 80
cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-xak8dfvt",source="test.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="+Inf"} 80
cloudprovider_azure_op_duration_seconds_sum{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-xak8dfvt",source="test.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 2690.9002259449994
cloudprovider_azure_op_duration_seconds_count{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-xak8dfvt",source="test.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 80
# HELP cloudprovider_azure_op_failure_count [ALPHA] Number of failed Azure service operations
# TYPE cloudprovider_azure_op_failure_count counter
cloudprovider_azure_op_failure_count{request="azuredisk_csi_driver_controller_delete_volume",resource_group="kubetest-xak8dfvt",source="test.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 1
cloudprovider_azure_op_failure_count{request="azuredisk_csi_driver_controller_publish_volume",resource_group="kubetest-xak8dfvt",source="test.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 63
# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 4.12e-05
... skipping 103 lines ...
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 7.6386304e+08
# HELP process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.
# TYPE process_virtual_memory_max_bytes gauge
process_virtual_memory_max_bytes -1
make: *** [Makefile:254: e2e-test] Error 1
2021/10/02 15:17:26 process.go:155: Step 'make e2e-test' finished in 39m50.988371253s
2021/10/02 15:17:26 aksengine_helpers.go:425: downloading /root/tmp2688293530/log-dump.sh from https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh
2021/10/02 15:17:26 util.go:68: curl https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh
2021/10/02 15:17:26 process.go:153: Running: chmod +x /root/tmp2688293530/log-dump.sh
2021/10/02 15:17:26 process.go:155: Step 'chmod +x /root/tmp2688293530/log-dump.sh' finished in 1.003531ms
2021/10/02 15:17:26 aksengine_helpers.go:425: downloading /root/tmp2688293530/log-dump-daemonset.yaml from https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump-daemonset.yaml
... skipping 53 lines ...
ssh key file /root/.ssh/id_rsa does not exist. Exiting.
2021/10/02 15:18:20 process.go:155: Step 'bash -c /root/tmp2688293530/win-ci-logs-collector.sh kubetest-xak8dfvt.westeurope.cloudapp.azure.com /root/tmp2688293530 /root/.ssh/id_rsa' finished in 2.55304ms
2021/10/02 15:18:20 aksengine.go:1141: Deleting resource group: kubetest-xak8dfvt.
2021/10/02 15:25:31 process.go:96: Saved XML output to /logs/artifacts/junit_runner.xml.
2021/10/02 15:25:31 process.go:153: Running: bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}"
2021/10/02 15:25:31 process.go:155: Step 'bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}"' finished in 169.022522ms
2021/10/02 15:25:31 main.go:331: Something went wrong: encountered 1 errors: [error during make e2e-test: exit status 2]
+ EXIT_VALUE=1
+ set +o xtrace
Cleaning up after docker in docker.
================================================================================
Cleaning up after docker
1b0d8fae32e7
... skipping 4 lines ...